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Preface to the Classics Edition 


During the fifteen years since the first edition of A First Course in Order Statis- 
tics appeared, there have been more than 1500 papers published relating to 
order statistics (based on a key word search in MathSciNet), i.e., about 100 
papers a year. We take this as strong evidence that basic knowledge of the the- 
ory of order statistics is, more than ever, a requisite component in the toolbox 
of any practicing statistician and an essential part of the training requirements 
for undergraduate and graduate students in the field. This edition of the First 
Course will, we believe, provide sufficient background knowledge to enable 
the reader to understand much of the current order statistics literature. We hope 
that it will prepare the next generation of students and researchers to contribute 
to the burgeoning flow of fascinating new results concerning ordered statistical 
data. We are grateful to the many alert readers who identified typographical 
and technical errors in the first edition of this book. We will continue to be 
grateful for any reports on further gaffes that have not been noted and included 
in the errata list that accompanies this second edition. 

For more extensive coverage of the field of order statistics, we continue to 
recommend that readers consult the latest edition of H. A. David’s Order Statis- 
tics, which, as of this writing, is the third edition, coauthored by H. N. Nagaraja. 
We also list a few other resources for further reading. 

As we did in the first edition, we urge readers to explore the realm of order 
statistics. We continue to be confident that once you bite into it, you will be 
hooked for life. 


Barry C. Arnold, Riverside, California 
N. Balakrishnan, Hamilton, Ontario 
H. N. Nagaraja, Columbus, Ohio 

October 25, 2007 
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Preface 


Years ago the study of order statistics was a curiosity. The appearance of 
Sarhan and Greenberg’s edited volume in 1962, and H. A. David’s treatise on 
the subject in 1970 and its subsequent revision in 1981 have changed all that. 
Some introduction to the topic is surely needed by every serious student of 
statistics. A one-quarter course at the advanced undergraduate level or 
beginning graduate level seems to be the norm. Up until now the aforemen- 
tioned book by Professor David has been used both as a text and as a 
growing survey and introduction to the by now enormous order statistics 
literature. It has served well as a text, but the task grows more daunting as 
the literature grows. To be an encyclopedia or to be a text, that is the 
question. We believe that it is time to write an introductory text in the area. 

There are certain topics that should and can be introduced in a one-quarter 
course. The coverage should be reasonably rigorous but not exhaustive. The 
David encyclopedia is available on the shelf, and students will expect to have 
frequent recourse to it. However, we have striven to write a textbook in the 
sense that at the end of the course the instructor can justifiably expect that 
the students will understand and be familiar with everything in the book. It 
should provide a springboard into further applications and research involving 
order statistics, but it should be restricted to “what everyone needs to know.” 

We have prepared this text bearing in mind students in mathematical 
sciences at the first-year graduate and advanced undergraduate levels. The 
prerequisite is a two-quarter or semester course sequence in introductory 
mathematical statistics. We have been successful in covering most of the 
topics discussed in this book in a one-quarter or semester course at our 
respective institutions. We hope this treatment will also interest students 
considering independent study and statisticians who just want to find out 
some key facts about order statistics. The exercises at the end of chapters 
have been designed to let the students have hands-on experience to help 
them develop a better understanding and appreciation of the concepts 
discussed in the text. They are naturally of varying degrees of difficulty; for 
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the more challenging exercises we have included the relevant references to 
assist the disheartened student. Improvements to our presentation are always 
possible and we welcome any suggestions or corrections from the users of this 
book. 

We have not tried to uniformly assign priority for published results. When 
references are given, it is with the understanding that they may be illuminat- 
ing. Most of the material in the book is in the order statistics “public 
domain” by now. A few new results or new view points are sprinkled in, and 
a few, hopefully, tantalizing questions are left open. 

If we have been successful, the student on completing this introductory 
volume will cry out for more. There is, of course, much more. That is why this 
is a first course. A sequel to this book is at the planning stage. It will 
hopefully probe further, building on the present material, exploring fascinat- 
ing byways which perhaps not everyone needs to know. 

The study of order statistics is habit forming. We hope this book will 
spread the addiction. 


Barry C. Arnold 
N. Balakrishnan 
H. N. Nagaraja 
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CHAPTER 1 


Introduction and Preview 


1.1. ORDER STATISTICS 

Suppose that (X l ,...,X„) are n jointly distributed random variables. The 
corresponding order statistics are the A,’s arranged in nondecreasing order. 
The smallest of the X/s is denoted by X 1:n , the second smallest is denoted 
by X 2; and, finally, the largest is denoted by X„.„. Thus X l:n < X 2 .„ 
<, < X n n . The focus of the present book is on the study of such 

so-called variational sequences. A remarkably large body of literature has 
been devoted to the study of order statistics. Developments through the early 
1960s were synthesized in the volume edited by Sarhan and Greenberg 
(1962a), which, because of its numerous tables, retains its usefulness even 
today. Harter (1978-1992) has recently prepared an eight-volume annotated 
bibliography with an excess of 4700 entries. Needless to say, we will need to 
exercise a degree of selectivity in preparing an introductory text. Our goal is 
to introduce the student to several of the main themes in the order statistics 
literature. After reading this introductory text, it is hoped that some readers 
will be enticed to seek more detailed and more comprehensive discussion of 
the topics. H. A. David’s (1981) masterful encyclopedic survey is a wonderful 
starting point. Fascinating related odysseys are described in Galambos (1978), 
Harter (1970), Barnett and Lewis (1984), Castillo (1988), and Balakrishnan 
and Cohen (1991). Our intention is to whet the appetite; the above authors 
will serve full-course meals. 

In the definition of order statistics we did not require that the A,’s be 
identically distributed, nor did we require that they be independent. Addi- 
tionally, it was not assumed that the associated distributions be continuous, 
nor that densities need exist. Many of the classical results dealing with order 
statistics were originally derived in more restrictive settings. It was often 
assumed that the A,’s were independent and identically distributed (hence- 
forth i.i.d.) with common continuous (cumulative) distribution function Fix), 
and often assumed further to have a density function fix). Undoubtedly 
most readers first encountered order statistics with such assumptions. For 
many, the first encounter with order statistics was doubly painful in that, not 
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only was the concept novel, but the derivation of the joint density of the 
order statistics was used as a convenient example of the use of Jacobians in a 
many-to-one mapping situation. No wonder the phrase “order statistics” gets 
few smiles from those emerging from the Hogg and Craig (1978), Mood, 
Graybill, and Boes (1974), Bickel and Doksum (1977), Dudewicz and Mishra 
(1988), and Caselia and Berger (1990) trenches. The end result of that 
analysis, confusing though it seemed at the time, was a disarmingly attractive 
expression for the joint density of the order statistics corresponding to an 
i.i.d. sample from an absolutely continuous distribution with density fix), 
namely, 

n 

f Xt . x m i x u*i x n) -mnn*,). 

i—i 

-00<Jt 1 <Jt 2 < ••• <Jt„<00. (1.1.1) 

Much of the material in the present book takes (1.1.1) as a starting point. In 
several instances results are available for non-i.i.d. or nonabsolutely continu- 
ous distributions. These will be appropriately noted. As much as possible, we 
will try to relax the i.i.d. and/or absolute continuity conditions, but the 
model described by (1.1.1) is the most easily visualized. For example, no 
tantalizing ties bedevil one in such settings. 

There is an alternative way to visualize order statistics that, although it 
does not necessarily yield simple expressions for the joint density (which may 
of course not even exist), does allow simple derivation of many important 
properties of order statistics. It can be called the quantile function represen- 
tation. In this setting we do consider i.i.d. A/s, but we allow their common 
distribution F to be completely arbitrary, that is, just nondecreasing and 
right continuous with Fi — °°) = 0 and F(°°) = 1. The associated quantile 
function (or inverse distribution function, if you wish) is defined by 

F“'(y) = sup{x: F(x) iSy}. (1.1.2) 

Now it is well known that if U is a Uniform(0, 1) random variable, then 

F~KU ) has distribution function F. Moreover, if we envision U x U n as 

being i.i.d. Uniform (0, 1) random variables and X U X 2 X„ as being i.i.d. 

random variables with common distribution F, then 


(X Un ,...,X n:n ) ± (F - 1 ((/,.„),..., F - '((/„.„)), (1.1.3) 

where = is to be read as “has the same distribution as.” In the absolutely 
continuous case, the representation (1.1.3) allows one to derive the joint 
density (1.1.1) from the corresponding simpler joint density of uniform order 
statistics, namely, 


fill, u„)=n\, 0<m,< ••• <«„<!• (1-1-4) 


In the nonabsolutely continuous case, the representation (1.1.3) frequently 
permits simple derivations of moments, mixed moments, and other distribu- 



ORDER STATISTICS 


3 


tional features of order statistics. Thus, for example, 

E(X hn ) = f'F-'WfyJ^du. (1.1.5) 

J o 

The representation X i:n = F~ l (U i:n ) is also useful in deriving the asymp- 
totic distribution of X i:n . More details will be found in Chapter 8, but 
a hint of the possibilities is provided by the following observations. Consid- 
er X lnp\:n where p e (0, 1) and n is large (here [np] denotes the integer 
part of np). Elementary computations beginning with (1.1.1) yield U i:n ~ 
Beta (i,n + 1 - /). If i = [np], then U lnp y „ ~ Beta(np, n(l - p)), and this 
Beta random variable has the same distribution as ^/(E". x V t + E? =1 H^), 
where the I^’s are i.i.d. T(p, 1) and the H^’s (independent of the I^’s) are i.i.d. 
T(1 - p, 1). An application of the multivariate central limit theorem and the 
delta method [see Bishop, Fienberg, and Holland (1975, Chapter 14)] yields 

Ulnpl-.n ~ N^p,~p{\ ~p) j. 


From this, if we assume that F has a density /, the delta method again may 
be applied to yield 


X[np]: n ~ ^ 


F-'(P), 


1 

n 


p{\-p) \ 

[/(f-v))] 2 J 


Extreme order statistics of course will have different nonnormal limiting 
distributions. Tippett (1925) may have been the first to guess the nature of 
such limiting distributions. The rigorous development of the solution is 
to be found in Fisher and Tippett (1928) and Gnedenko (1943). Anticipating 
again material in Chapter 8 we can guess that the nature of the distribution 
of X n n will depend only on the upper tail of F. If, for example, F(x) = 
1 - Fix) ~ ac"“[x -» °o], then we can argue as follows: 


P{b n X n:n <x)=P\X n:n <y 


X 

F I- 


1 — FI A 


1 - 


If we then choose b„ = n l/a , then the last expression becomes [1 - x a /n] n . 
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which converges to e~ x “, often called the extreme value distribution of the 
Frech€t type. 

Rather than continue to anticipate theoretical results to be introduced in 
later chapters, it is undoubtedly time to convince the reader that order 
statistics are of more than theoretical interest. Why have statisticians cared 
about order statistics? In what data-analytic settings do they naturally arise? 
An array of such situations will be catalogued in the next section. 


1.2. HERE, THERE, AND EVERYWHERE 

When asked to suggest a brief list of settings in which order statistics might 
have a significant role, none of our colleagues had difficulty; the only problem 
was that their lists were not always brief! The following list, culled from their 
suggestions, is not exhaustive, but should serve to convince the reader that 
this text will not be focusing on some abstract concepts of little practical 
utility. Rather, the reader may realize that in some cases it will just be new 
names for old familiar concepts, and will share the delight of the man 
entranced to discover he had been speaking prose all of his life. 

1. Robust Location Estimates. Suppose that n independent measurements 
are available, and we wish to estimate their assumed common mean. It has 
long been recognized that the sample mean, though attractive from many 
viewpoints, suffers from an extreme sensitivity to outliers and model viola- 
tions. Estimates based on the median or the average of central order 
statistics are less sensitive to model assumptions. A particularly well-known 
application of this observation is the accepted practice of using trimmed 
means (ignoring highest and lowest scores) in evaluating Olympic figure 
skating performances. 

2. Detection of Outliers. If one is confronted with a set of measurements 
and is concerned with determining whether some have been incorrectly made 
or reported, attention naturally focuses on certain order statistics of the 
sample. Usually the largest one or two and/or the smallest one or two are 
deemed most likely to be outliers. Typically we ask questions like the 
following: If the observations really were i.i.d., what is the probability that the 
largest order statistic would be as large as the suspiciously large value we 
have observed? 

3. Censored Sampling. Fifty expensive machines are started up in an 
experiment to determine the expected life of a machine. If, as is to be hoped, 
they are fairly reliable, it would take an enormously long time to wait for all 
machines to fail. Instead, great savings in time and machines can be effected 
if we base our estimates on the first few failure times (i.e., the first few order 
statistics from the conceptual sample of i.i.d. failure times). Note that we may 
well remove unfailed machines from the testing environment for sale and/or 
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modification. Only the failed items are damaged and/or destroyed. Such 
procedures are especially important from a destructive testing viewpoint 
(how many Mercedes-Benz motors should you burn up in oil breakdown 
experiments?). From an ethical viewpoint, even more pressure is put on the 
medical researcher in his or her efforts to learn about the efficacy of 
treatments with a minimum of actual (patient) failures. 

4. Waiting for the Big One. Disastrous floods and destructive earthquakes 
recur throughout history. Dam construction has long focused on so called 
100-year floods. Presumably the dams are built big enough and strong enough 
to handle any water flow to be encountered except for a level expected to 
occur only once every 100 years. Architects in California are particularly 
concerned with construction designed to withstand “the big one,” presum- 
ably an earthquake of enormous strength, perhaps a “100-year quake.” 
Whether one agrees or not with the 100-year disaster philosophy, it is obvious 
that designers of dams and skyscrapers, and even doghouses, should be 
concerned with the distribution of large order statistics from a possibly 
dependent, possibly not identically distributed sequence. 

5. Strength of Materials. The adage that a chain is no stronger than its 
weakest link underlies much of the theory of strength of materials, whether 
they be threads, sheets, or blocks. By considering failure potential in in- 
finitesimally small sections of the material, one quickly is led to strength 
distributions associated with limits of distributions of sample minima. Of 
course, if we stick to the finite chain with n links, its strength would be the 
minimum of the strengths of its n component links, again an order statistic. 

6. Reliability. The example of a cord composed of n threads can be 
extended to lead us to reliability applications of order statistics. It may be 
that failure of one thread will cause the cord to break (the weakest link), but 
more likely the cord will function as long as k (a number less than n) of the 
threads remain unbroken. As such it is an example of a k out of n system 
commonly discussed in reliability settings. With regard to tire failure, the 
automobile is often an example of a 4 out of 5 system (remember the spare). 
Borrowing on terminology from electrical systems, the n out of n system is 
also known as a series system. Any component failure is disastrous. The 1 out 
of n system is known as a parallel system; it will function as long as any of the 
components survives. The life of the k out of n system is clearly X n _ k + l:n , 
the (n - k + l)st largest of the component lifetimes, or, equivalently, the 
time until less than k components are functioning. Other more complicated 
system structures can be envisioned. But, in fact, they can be regarded as 
perhaps complicated hierarchies of parallel and series subsystems, and the 
study of system lifetime will necessarily involve distributions of order statis- 
tics. 

7. Quality Control. Take a comfortable chair and watch the daily produc- 
tion of Snickers candy bars pass by on the conveyor belt. Each candy bar 
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should weigh 2.1 ounces; just a smidgen over the weight stated on the 
wrapper. No matter how well the candy pouring machine was adjusted at the 
beginning of the shift, minor fluctuations will occur, and potentially major 
aberrations might be encountered (if a peanut gets stuck in the control 
valve). We must be alert for correctable malfunctions causing unreasonable 
variation in the candy bar weight. Enter the quality control man with his X 
and R charts or his median and R charts. A sample of candy bars is weighed 
every hour, and close attention is paid to the order statistics of the weights so 
obtained. If the median (or perhaps the mean) is far from the target value, 
we must shut down the line. Either we are turning out skinny bars and will 
hear from disgruntled six-year-olds, or we are turning out overweight bars 
and wasting money (so we will hear from disgruntled management). Atten- 
tion is also focused on the sample range, the largest minus the smallest 
weight. If it is too large, the process is out of control, and the widely 
fluctuating candy bar weights will probably cause problems further down the 
line. So again we stop and seek to identify a correctable cause before 
restarting the Snickers line. 

8. Selecting the Best. Field trials of corn varieties involved carefully bal- 
anced experiments to determine which of several varieties is most productive. 
Obviously we are concerned with the maximum of a set of probably not 
identically distributed variables in such a setting. The situation is not unlike 
the one discussed earlier in the context of identification of outliers. In the 
present situation, the outlier (the best variety) is, however, good and merits 
retention (rather than being discarded or discounted as would be the case in 
the usual outliers setting). Another instance in biology in which order 
statistics play a clear role involves selective breeding by culling. Here perhaps 
the top 10% with respect to meatiness of the animals in each generation are 
retained for breeding purposes. A brief discussion of this topic will be found 
in Section 8.6. 

9. Inequality Measurement. The income distribution in Bolivia (where a 
few individuals earn most of the money) is clearly more unequal than that of 
Sweden (where progressive taxation has a leveling effect). How does one 
make such statements precise? The usual approach involves order statistics 
of the corresponding income distributions. The particular device used is 
called a Lorenz curve. It summarizes the percent of total income accruing to 
the poorest p percent of the population for various values of p. Mathemati- 
cally this is just the scaled integral of the empirical quantile function, a 
function with jump X hn at the point i/n\ i = 1,2 ,...,n (where n is the 
number of individual incomes in the population). A high degree of convexity 
in the Lorenz curve signals a high degree of inequality in the income 
distribution. 

10. Olympic Records. Bob Beamon’s 1968 long jump remains on the 
Olympic record book. Few other records last that long. If the best perfor- 
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mances in each Olympic Games were modeled as independent identically 
distributed random variables, then records would become more and more 
scarce as time went by. Such is not the case. The simplest explanation 
involves improving and increasing populations. Thus the 1964 high jumping 
champion was the best of, say, N, active international-caliber jumpers. In 
1968 there were more high-caliber jumpers of probably higher caliber. So we 
are looking, most likely, at a sequence of not identically distributed random 
variables. But in any case we are focusing on maxima, that is, on certain 
order statistics. More details on record values and in particular record values 
in improving populations will be found in Chapter 9. Of course, such models 
actually make the survival of Beamon’s record even more problematic. 

11. Allocation of Prize Money in Tournaments. At the end of the annual 
Bob Hope golf tournament the player with the lowest score gets first prize. 
The second lowest score gets second prize, etc. In 1991 the first five prizes 
were: $198,000, $118,800, $74,800, $52,800, and $44,000. Obviously we are 
dealing with order statistics here. Presumably the player with the highest 
ability level will most likely post the lowest score. But, of course, random 
fluctuations will occur, and it is reasonable to ask what is the most equitable 
way to divide the prize money among the low-scoring golfers. Winner take all 
is clearly not in vogue, so some monotonically decreasing sequence of 
rewards must and has been determined. Is the one used a fair one? A related 
question involves knockout tournaments. In the absence of seeding, and 
assuming that a stronger player always beats a weaker player, the ability rank 
of the winner is always 1, but the ability rank of the loser in the final will 
often be greater than 2 (as the second best player may well have been 
knocked out in an earlier round by the eventual winner.) Again, we may ask 
what is an equitable reward system for such a tournament. Any discussion of 
complete or incomplete tournaments will involve consideration of order 
statistics. 

12. Characterizations and Goodness of Fit. The exponential distribution is 
famous for its so-called lack of memory. The usual model involves a light 
bulb or other electronic device. The argument goes that a light bulb that has 
been in service 20 hours is no more and no less 'ikely to fail in the next 
minute than one that has been in service for, say, 5 hours, or even, for that 
matter, than a brand new bulb. Such a curious distributional situation is 
reflected by the order statistics from exponential samples. For example, if 
X u ...,X n are i.i.d. exponential, then their spacings (.X i: „ - X i _ l .„) are 
again exponential and, remarkably, are independent. It is only in the case of 
exponential random variables that such spacings properties are encountered. 
A vast literature of exponential characterizations and related goodness-of-fit 
tests has consequently developed. We remark in passing that most tests of 
goodness of fit for any parent distribution implicitly involve order statistics, 
since they often focus on deviations between the empirical quantile function 
and the hypothesized quantile function. 




CHAPTER 2 


Basic Distribution Theory 


2.1. INTRODUCTION 

In this chapter we discuss in detail the basic distribution theory of order 
statistics by assuming that the population is absolutely continuous. Many of 
these derivations for the case when the population is discrete are handled in 
Chapter 3. 

In Section 2.2 we derive the cumulative distribution function and probabil- 
ity density function of a single order statistic, X i:n . In Section 2.3 we similarly 
derive the joint cumulative distribution function and joint density function of 
two order statistics, X i:n and X j n . Due to the importance of the results 
given in Sections 2.2 and 2.3, we also present some interesting and useful 
alternate derivations of these formulas. In Section 2.4 we establish some 
distributional properties of order statistics and show, in particular, that the 
order statistics form a Markov chain. In Section 2.5 we discuss the derivation 
of the distributions of some specific statistics such as the sample median, the 
range, the quasiranges, and the spacings or the generalized quasiranges. We 
make use of the uniform and the power-function distributions in order to 
illustrate the various results developed in this chapter. 


2.2. DISTRIBUTION OF AN ORDER STATISTIC 

As mentioned in the last chapter, let us assume that X V X 2 X n is a 

random sample from an absolutely continuous population with probability 
density function (density function pdf) f(x ) and cumulative distribution 
function (cdf) F(x); let X l:n ^ X 2 . n <. ••• £X„.„ be the order statistics 
obtained by arranging the preceding random sample in increasing order of 
magnitude. Then, the event x < X i: „ £x + Sx is essentially same as the 
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following event: 


i-l 1 n-i 

]— ] ; 

- 00 X X + Sx 00 

X r <, x for i - 1 of the X r ’s, x < X r < x + Sx for exactly one of the X r ’s, 
and X r > x + Sx for the remaining n - i of the X r ’s. By considering Sx to 
be small, we may write 


p( X <x i:n <x+sx) = ~ n _ i} j { F ( X )r'{i- F (x + s*)r' 

X{F( x + Sx) - /’(x)} + 0((5x) 2 ), (2.2.1) 

where 0((8x) 2 ), a term of order (Sx) 2 , is the probability corresponding to 
the event of having more than one X r in the interval (x, x + Sx]. From 
(2.2.1), we may derive the density function of X i:n (1 < / <, n) to be 


f i:n (x)= lim 

ajr-<0 


P(x < X i:n < x + Sx) 
Sx 


n\ 

(«■ - l)!(n - /)! 




F(x)} n -‘f(x), 

-» < x < ». (2.2.2) 


The above expression of the pdf of X i:n can also be derived directly from 
the joint density function of all n order statistics. To show this we first of all 
need to note that, given the realizations of the n order statistics to be 
x t .„ < x 2 :„ < ■ • • the original variables X, are restrained to take on 

the values x i:n (/ = 1,2,..., n), which by symmetry assigns equal probability 
for each of the n\ permutations of (1,2, ... , n). Hence, we have the joint 
density function of all n order statistics to be 


/i. 


2 n:n 


(x x ,x 


2 » • 


-»!n/(*r). -°°<*1<*2< 

r-1 


< x n < «- 
(2.2.3) 


The above joint density function of all n order statistics may be derived 
alternatively using the Jacobian method and a n!-to-l transformation as 
described in Hogg and Craig (1978); see also Chapter 1 for an argument 
based on uniform order statistics. Thus, for example, in the case of standard 
uniform distribution with density function f(u) = 1, 0 ^ u £ 1, and standard 
exponential distribution with density function fix) = e~ x , 0 <,x < », we 
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have the joint density function of all n order statistics to be 

f 1,2 «:«(“!. “2. •■•.«„) =«!, 0<«,<« 2 < • • <U n < 1, (2.2.4) 

and 

/l,2 n-.n{x x ,x 2 ,...,x n ) = n\e~ Z "-' X ', 0 < x, < x 2 < ■■■ <X n < 00 , 

(2.2.5) 

respectively. We will make use of the above joint density functions later in 
Chapter 4 in order to establish some properties of uniform and exponential 
order statistics. For example, from (2.2.4) we will show later in Section 4.7 
that L( :n /L^. n and U j:n (1 < i < j < n) are independently distributed as 
BetaO', j - i) and Beta (/', n - j + 1), respectively. Similarly, from the joint 
density function of exponential order statistics in (2.2.5) we will show later in 
Section 4.6 that the set of exponential spacings X i :n - A r i _ 1 (with X 0:n = 0) 
form a set of n independent exponential random variables. 

Now, by considering the joint density function of \ all n order statistics 
in Eq. (2.2.3) and integrating out the variables (A', . . . , X i _ l .„) and 
(X i+l n , . . . , X n n \ we derive the marginal density function of X i n 
(1 < i < rt) to be 

fi.n(x) = «!/(*)(/ " j • • f( x i-i)d*i • <k/-ij 

X {L " ' i x f( x n)d x i + \ •• (2.2.6) 

Direct integration yields 


/ ••• J’ j • • /(^,-l)^1^2 


dx 


1-1 


- OO —00 

i i- 1 


{F(x)}'-7(/-1)! 


(2.2.7) 


and 


/ • • • / ' + 7 , *V(JCi+i)/(JCf+ 2 ) • ■ • f(x n ) <fr, + i ^, + 2 • • • dx n 

X * X J X 

-{1-F(x)}"“7(« -«)!. (2.2.8) 

Substitution of the expressions (2.2.7) and (2.2.8) for the two sets of integrals 
in Eq. (2.2.6) gives the pdf of X i n (1 < i < n) to be exactly the same 
expression as derived in (2.2.2). 
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The pdfs of the smallest and largest order statistics follow from (2.2.2) 
(when / = 1 and / = n) to be 

/,:„(*) = n{\ - F(x)) n -'f(x), -oe <* < oo, (2.2.9) 
and 

f„M) = n{F(x)} n -'f(x), -°c<x< oo, (2.2.10) 


respectively. 

The distribution functions of the smallest and largest order statistics are 
easily derived, by integrating the pdfs in (2.2.9) and (2.2.10), to be 


F i:n (x) = 1 - {1 - *•(*)}", (2.2.11) 

and 

F„ : „(*) = {F(-0} fl , -°°<*<*. (2.2.12) 

In general, the cdf of X jn may be obtained by integrating the pdf of X i:n in 
(2.2.2). It may also be derived without much difficulty by realizing that 


F i:n (x) = P(X i:n <x) 

= P(at least / of X x , X 2 ,. . . , X n are at most x ) 

n 

= £ P(exactly r of X,, X 2 , . . . , X n are at most x) 

r = i 

= E (?){ /r (^)} r { 1 - F (x)) n -<» < x <oo. (2.2.13) 

r = i 


Thus, we find that the cdf of X i n (1 < / < n) is simply the tail probability 
(starting from /') of a binomial distribution with Fix) as the probability of 
success and n as the number of trials. We may note that F'. n (x) in (2.2.13) 
reduces to the expressions in (2.2.11) and (2.2.12) when /' = 1 and / = n, 
respectively. Furthermore, by using the identity that 





n\ 

!)!(/. -/)! 


r'~'(l - t) n 


‘ dt. 


0 <p < 1 

(2.2.14) 


(which may be proved easily by repeated integration by parts), we can write 
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the cdf of X,.„ from (2.2.13) equivalently as 



= l,. lx) (i,n - i + 1), -oc<jf<x, (2.2.15) 

which is just Pearson’s (1934) incomplete beta function. It should be pointed 
out here that the expression of Fj.„(x) in (2.2.15) holds for any arbitrary 
population whether continuous or discrete. However, under the assumption 
that the population is absolutely continuous, we may differentiate the expres- 
sion for the cdf in (2.2.15) and derive the pdf of X i n (1 < i < n) to be 
exactly the same expression as given in (2.2.2). 

It is important to mention here that one can write the cdf of A in terms 
of negative binomial probabilities as noted by Pinsker, Kipnis, and 
Grechanovsky (1986), instead of the binomial form given in (2.2.13). To see 
this, let us write 

F i:ii (x) = P(X, :ii <x) 

= P (reaching i successes in the course of at most n trials with 
probability of success Fix)) 

= -f(x)}' 

= L (” • _! . r ){^U)} {> - F ( x )} - -x<jr<3c. 

<■ = n ' > 


Example 2.2.1. Let us consider the standard uniform distribution 
with density function fin) = 1. 0 < it < 1, and cdf Fin) = u. 0 < it < 1. 
Then, from Eqs. (2.2.13) and (2.2.15), we immediately have the cdf of U i . n 
(1 < i < n) to be 


r~r 1 ('-!)!(« 


- »)! 


f'-'(i - o" ' (it. 


()<»<!. (2.2.16) 


From Eq. (2.2.2), we have the density function of U i:H ( 1 < i < n) to be 

rl 


//:„(«) = 


/I ! 


(i - !)!(«-»)! 


u'~ ] ( 1 - u) 


()<»<!. (2.2.17) 
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From the above density function, we obtain the mth moment of U i:n to be 

,47>=£(^?J =/>£,>) du 

J o 

= B(i + m,n - i + \)/B{i, n - i + 1), 
where B(-,-) is the complete beta function defined by 

B(p,q) = -t) q ~ l dt, p,q> 0, (2.2.18) 

■'o 

which yields upon simplification 


n! (i + m-1)! 
(n+m)] (i — 1)! 


(2.2.19) 


From Eq. (2.2.19) we obtain, in particular, that for / = 1,2, .... n 

p i:n = E(U i:n )= Pi (2.2.20) 

and 

= oj® = var(^ :n ) = - p), n = pfii/ ( n + 2), (2.2.21) 

where p, = i/(n + 1) and q i , = 1 - p t . From the pdf of U i:n (1 ^ i < n ) in 
(2.2.17), we may also easily work out the mode of the distribution of U i:n to 
be at (i - l)/(n - 1). 


Example 2.2.2. Next let us consider the standard power-function distri- 
bution with density function f(x) = vx v ~ l , 0 < x < 1, v > 0, and cdf 
F(x) = x v , 0 < x < 1, v > 0. From Eqs. (2.2.13) and (2.2.15), we then have 
the cdf of X i n (1 < / <, n) to be 


FU*)= L( n r )(x v Y(l- xV ) n ~ r 

r=i 



n\ 

(«• -!)!(«-«)! 


r‘- l (l 


0 <x < 1, v > 0. 

( 2 . 2 . 22 ) 


From Eq. (2.2.2), we similarly have the density function of X rn (1 £n) 
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to be 


/,:„(*) = 


n\ 


(i -!)!(/. -/)! 


vx ,v -\l -x v ) 


0 < X < 1, F > 0 

(2.2.23) 


from which we obtain the mth moment of X i:n (1 < i <, n ) to be 


nS? n =E(X” n ) = fx m f i:n (x)dx 
J o 

= + — , n - i + lj n — i + 1) 


T(n + 1) r(i + m/v) 
T(n + 1 + m/v) r(i) 

where IX ■ ) is the complete gamma function defined by 


(2.2.24) 


T(p) = r e -'t p ~ 1 dt, p> 0. (2.2.25) 

■'o 

In particular, by setting m = 1 and m = 2 in Eq. (2.2.24), we obtain 


and 


Hr.n-E(X i:n ) 


T(n + 1) r(i + l/v) 

r(n + 1 + l/v) f(7) 


(2.2.26) 




r(« + i) r(/ + 2/v) 
r(« + i + 2/v) r(7j 


(2.2.27) 


from which we get 
= var(Jf i: „) 

r(« + i) ( r(i + 2/v) r (/ + i/ v ) \ 

r(i ) ( r(n + 1 + 2/v) ~ T(n + 1 + l/v) ^' :n j ( 2 - 2 - 28 ) 


Further, from the pdf of X i:n in (2.2.23), we observe that the mode of the 
distribution of X i:n is at ((iv — \)/(nv — l)) 1/l '. We may also note that the 
results obtained in Example 2.2.1 for the uniform order statistics can all be 
deduced from here by setting v = 1. 
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2.3. JOINT DISTRIBUTION OF TWO ORDER STATISTICS 

In order to derive the joint density function of two order statistics X i:n and 
Xj. n (1 < / <j < n), let us first visualize the event (x, < X i:n < x, + Sx,, 
x j < X j n < Xj + 8xj) as follows: 


- X Xj Xj + Sx, Xj Xj + &x ) x 

X r < Xj for i — 1 of the X r 's, x, <X r < x , + Sx, for exactly one of the X r 's, 
Xj + 8Xj <X r < xj for j - i - 1 of the X r ’s, x y <X r < x y + 8x } for exactly 
one of the X r 's, and X r > x ; + 8xj for the remaining n - j of the X \'s. By 
considering Sx, and 8xj to be both small, we may write 

P( x i x i + 8x n X , < X j:n * x j + 8 x j) 

x{F(x y ) - F(Xj + Sx,)p~'{l - F( Xj + Sx,)}'"' 

x{F(x, + Sx,-) - F(x,-)}{F(xy + Sx,-) - F(x,)} 

+ o((Sx f .) 2 Sx > ) + o(Sx,(Sx,) 2 ); (2.3.1) 

here 0((8Xj) z 8xj) and 0(Sx,(Sx,) 2 ) are higher-order terms which corre- 
spond to the probabilities of the event of having more than one X r in the 
interval (x„ x t + Sx,] and at least one X r in the interval (x,, x, + Sx,], and 
of the event of having one X r in (x,, x, + Sx,] and more than one X r in 
(x y , Xj + Sx,], respectively. From (2.3.1), we may then derive the joint density 
function of X f .„ and Xj.„ (!</<./< n) to be 


/i. j :»(*/.*>) 


lim 

8x j — * 0 . 8xj — > 0 


P(x,- < Xj.„ < Xj + 8 Xj, Xj <Xj. n <ZXj + 8 Xj) 


8Xj8Xj 


n ' 


(|- - l)!(y - i - 1)!(« - j ) ! 

x {F(x,)} ( *{F(x,) - F(Xj)} y _l {l - F(Xj))" J f(Xj)f(Xj), 

-oc<x ,-<x,-<«. (2.3.2) 
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The joint density function of X i:ii and X j:n (1 < i <j < n) given in 
(2.3.2) can also be derived directly from the joint density function of all n 
order statistics as follows. By considering the joint density function of all n 
order statistics in Eq. (2.2.3) and then integrating out the variables 
(A' 1: „,..., :n ), ( X, Xj_ l: „), and ( X J + , X„.„), we derive 

the joint density function of X j: „ and Xj. n (1 < i < j < n) to be 

fij-.n( x hXj) =«!/U,)/Uy)(/_^--- /_7(*i> " f( x ,i)dx ] ••• dx,_ ,J 


x i 

{!"■■ 

■ / 7u,+i) • 

■ /(x y _, )<**,-+, <fr y _,J 



J x, 


f 

^ J + 1 

) 

x l 


7 7(^+i)-- 

• f( x „)dx j+l •• • (2.3.3) 


By direct integration we obtain 


/ 'll "/(*!)/( * 2 ) ■ • • /(*,■_,) dx, dx 2 

= {^K)r7(<-l)!, (2.3.4) 

('■■■( " ( '7(-U + i)/( •*, + :) ••• /(jf y _ l )rfr /+1 rfr | . + 2 dx j _ , 

*x, *\ry 

= {F(t ; ) - E(t,)} j ' */(/ — * — !)!, (2.3.5) 


/ " / ' 7 ,+ 7(x >+l )/(T y+2 ) ••• f(x„)dx j+l dx j+2 

Ty T, J X j 

= {1 -E(* y )}"~7(« -/)!. (2.3.6) 

Upon substituting the expressions (2.3.4)— (2.3.6) for the three sets of inte- 
grals in Eq. (2.3.3), we obtain the joint density function of X rn and X j . n 
0 < i <j < n) to be exactly the same expression as derived in (2.3.2). 

In particular, by setting i =1 and j = n in (2.3.2), we obtain the joint 
density function of the smallest and largest order statistics to be 

-o = "(« - i)[F ( x j - tu, ) r“ 2 /(*,)/K), 

-oc < Jf, < x„ < oo; (2.3.7) 
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similarly, by setting / = i + 1 in (2.3.2), we obtain the joint density function 
of two contiguous order statistics, X i:n and X i+l:n (1 <, i ^ n - 1), to be 


fi,i+ 1 :n( X i > X i+ l) 


n! 

(i - l)!(n - i - 1)! 

x {*■(*, or'f 1 

-*<x f <x f+ ,<*. (2.3.8) 


The joint cumulative distribution function of X i n and X j:n can, in 
principle, be obtained through double integration of the joint density func- 
tion of X i n and X j n in (2.3.2). It may also be written as 

Fi, i : „( >x,)=F i: „( Xj) for x, £ Xj , 


and for jc, < x p 

F i,y.n( X i>Xj ) = P(*, :n ^ X j n £Xj) 


= P (at least i of X x , X 2 X n are at most jc, and at least 

j of X x , X 2 X n are at most Xj) 


n s 

= £ D P(exactly r of X t ,X 2 X n are at most jc, and 

i-; r=« exactly s of X u X 2 X n are at most Xj) 


£4- nl 

S r\(s - r)\(n - s)\ 

*{f( *,))>(•>,) - *(*,))“ '{l - F (*, ))’"'• (M-») 


Thus, we find that the joint cdf of X i:n and X jn (l <. i < j <. n) is the tail 
probability [over the rectangular region (j, i ), (/, i + 1) (n, rt)] of a bivari- 

ate binomial distribution. By using the identity that 


££77 

r-i * 


n! 


(*-/•)!(« — s)! 


P[(P 2 -PiY (1 “Pa)' 


n Pl 

, (1- !)!(;- 1 -!)!(«-/)! 


Xt| (1 h) ' dt 2 dt l , 


0 <Pi <P 2 < 1. 

(2.3.10) 
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we can write the joint cdf of . n and X j . n in (2.3.9), equivalently, as 




rF(x,) tF(xj) 

'o \ (i - 1) !(y - i — 1) !(« -j)\ 


-00 < x, < Xj < 00 , (2.3.11) 


which may be noted to be an incomplete bivariate beta function. The 
expression of F, Xj) in (2.3.11) holds for any arbitrary population 
whether continuous or discrete. But, when the population is absolutely 
continuous, the joint density function of X i:n and X j n (1 < i < j < n ) in 
(2.3.2) may be derived from (2.3.11) by differentiating with respect to both x t 
and Xj. 


Example 2.3.1. As in Example 2.2.1, let us consider the standard uni- 
form population. In this case, from Eqs. (2.3.9) and (2.3.11) we have the joint 
cdf of \J i n and Uj.„ (1 < i < j < n) to be 


F i.j:n( u tfUj) = E E 


«! 


-jr-i r! ( S “ r )!(” -5 ) ! 


«.y r ( i 



n u i 

, (i- 1) KJ-i- !)!(« -JY- 

Xt', -1 (r 2 - t]Y~‘ *0 _ h)" ‘ dt 2 dt u 0<u ( <«y<l. 

(2.3.12) 


Similarly, from Eq. (2.3.2) we have the joint density function of and Uj. n 
(1 < i < j < n) to be 


//.J:»(“i. “y) 


(i -l)!(j -*•-!)!(«-;)! 


unuj-Uiy-'-'ii-Uj)"-’, 

0 < «, < ii ; < 1 . (2.3.13) 


From the above joint density function, we obtain the (m^m-Hh product 
moment of (U i:n , U j n ) to be 


(m.-.my) 
“i,; : n 



(«,, Uj) du i duj 


n\ 

(« - !)!()-« - l)!(n -j)\ 


XB(i + m t ,j - i)B(j + m, + nij,n - j + 1), 
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which upon simplification yields 

n! (/' 4- m ■ - 1)! (j + m , 4- m - 1)! 

( n + rrij + rrij )! ( i — 1 ) ! (y + m,-— 1)! 

In particular, by setting m, = = 1 in Eq. (2.3.14), we obtain 


(2.3.14) 




'O' + 1) 


1 ^ / < j < n, (2.3.15) 


(n + l)(n + 2)’ 
which, when used with the expression of in (2.2.20), gives 

= CO v(U i: „,U j: „) = = PiQj/( n + 2), (2.3.16) 


where, as before, p, = //(« 4- 1) and = l - Pj. 

Example 2.3.2. Let us take the standard power-function distribution 
considered already in Example 2.2.2. In this case, from Eqs. (2.3.9) and 
(2.3.11) we have the joint cdf of X i:n and Xj.„ (!</<)< n) to be 


Fi.j:*( x i’ x j) £ r i(s _ r )u n - s )\ Xi ( x > ^ 


<=jr=i r -( S - 0 ! (« - 0 ! 
t' fl ^ 

n x '* • 

,'(/- 1)!0-I - l)!(n -))! 

x/r l (/2-^r , ' l (i -t 2 r i dt 2 dt x . 


0 < jc, < Jfj < 1 , p > 0. (2.3.17) 


From Eq. (2.3.2), we can similarly get the joint density function of X i:n and 
Xj. n (1 < i <j < n). From that expression we immediately obtain 


= E(x^xp„) = jj . ,,( , x j ) dx.dX; 


nl 


(/- l)!(y - i - l)!(n -y).' 

/ mi \ ( rrij 4- m; \ 

xb(/ 4- — i,y - i)fl(y + -j + l)- ( 2 . 3 . 18 ) 
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This, upon simplification, yields 

= r(/i + 1) l\i + m t /v) r[j + (m, + m^/p] 

tl,J " r[n + 1 + (m, ■ + nij)/v] \'(i) r (j + nij/v) 

1 <i<j<n. (2.3.19) 

In particular, by setting m, = m j = 1 in Eq. (2.3.19), we obtain 

P(«+l) T'(f + 1 /p) r(y + 2/p) 

" ~ E ( X ‘ " X ‘ ") ~ ]’(„ + i + 2/v ) F(7) r(y + 1 / v ) ’ 

I < i < j < n. (2.3.20) 

When used with the expression of /x, . „ in (2.2.26), this gives 
= cov( A", ,,, X r „) 

T(n + 1) f V(j + 2/v) l'(i + 1/p) I'(, + 1/p) \ 

F(/) \ T() + I/p) T(« + 1 + 2/p) R,, + 1 + l/p)^ J 

1 </<)<«, p > 0. (2.3.21) 


2.4. SOME PROPERTIES OF ORDER STATISTICS 

Let (7,, U 2 , ■ ■ ■ , U n be a random sample from the standard uniform distribu- 
tion and X,, X 2 , . ■ ■ , X n be a random sample from a population with cdf 
Fix). Further, let f/ l: „<(/ 2 .„< ••• < U li:n and X l .„<X 2:n < ••• < 
X n n be the order statistics obtained from these samples. 

Specifically, when Fix) is continuous the probability integral transforma- 
tion U = F(X) produces a standard uniform distribution. Thus, when Fix) is 
continuous we have 


F{X,,,) = U i:n , «=1,2,...,«. (2.4.1) 

Further, with the inverse cumulative distribution function F~'i ) as de- 
fined in (1.1.2), it is easy to verify that 


F~'(Uj) = X n 1=1,2,...,/, 


for an arbitrary Fi ■ ). Since F '(•) is also order preserving, it immediately 
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follows that 


F~'(U i:n )ix i:n , i-l,2 n. (2.4.2) 

The distributional relations in (2.4.1) and (2.4.2) were originally observed by 
Scheffe and Tukey (1945). The relation in (2.4.2) could have been used along 
with the expression of the cdf of U i:n in (2.2.16) in order to derive the cdf 
and the density function of X i:n (l <. i <. n ) to be exactly as given in Eqs. 
(2.2.15) and (2.2.2), respectively. The single and the product moments of 
order statistics X i n can be obtained from Eqs. (2.2.2) and (2.3.2) as 

1 < i <, n, m = 1,2 


and 


(i - 1) !(/-/- 1) !(«-/)! 

x // x | !-'x ; ">{F(x l )}'- 1 {F(x > ) -F(x ,)}'"‘ _1 

- 0° <Xj <Xj < °° 

X (l - F(x j )} n ~ } f(x i )f(x j ) dXj dxj, 1 <.i<j < n,m h m, 2: 1. 


Alternatively, by using (2.4.2) they may also be written more compactly as 


1 £ i < n , m ^ 1 , 


and 


Xu{ -1 (uy - m,) /_,_1 (1 - Uj)" ’ diijduj. 


1 <.i <j <>n, m it ntj ^ 1. 


The distributional relation in (2.4.2) will also be utilized later on in Section 
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5.5 in order to develop some series approximations for the moments of order 
statistics X i:n in terms of moments of the uniform order statistics U i:n . 

In the following two theorems, we relate the conditional distribution of 
order statistics (conditioned on another order statistic) to the distribution 
of order statistics from a population whose distribution is a truncated form of 
the original population distribution function Fix). 

Theorem 2.4.1. Let X x , X 2 , ■ ■ ■ , X n be a random sample from an abso- 
lutely continuous population with cdf Fix ) and density function fix), and let 
X\. n ^X 2 < X n . n denote the order statistics obtained from this 
sample. Then the conditional distribution of X J:n , given that X i n = jc, for 
i < j, is the same as the distribution of the (j - Oth order statistic obtained 
from a sample of size n - i from a population whose distribution is simply 
Fix) truncated on the left at jc,. 

Proof. From the marginal density function of X i:n in (2.2.2) and the joint 
density function of X i:n and X j:n (1 < i <j < n) in (2.3.2), we have the 
conditional density function of X-. n , given that X i n = jc,, as 


fj:n{,Xj\^i:n -*i) ft, j :n( ) /fi :n( ) 

- (»-Q! ( FCxJ-FCx,.) ]' 

(j - i - 1) !(« -»! \ 1-FK) / 

xf l-^ P f(Xi) 

\l -F( Xi )j 1 ~F( Xi )’ 


i < j < n, Xj < x j < oo. (2.4.3) 

The result follows from (2.4.3) by realizing that {F(xj) - F(x t ))/{\ - F(jc,)} 
and f(xj)/{l — Fix,)} are the cdf and density function of the population 
whose distribution is obtained by truncating the distribution F(x) on the left 
at jc,. □ 

Theorem 2.4.2. Let X u X 2 , . . . , X n be a random sample from an abso- 
lutely continuous population with cdf F(x) and density function /(jc), and let 
Xi.n — ^2:n ^ ^ X n . n denote the order statistics obtained from this 

sample. Then the conditional distribution of X i n , given that Xj. n = x . for 
j > i, is same as the distribution of the ith order statistic in a sample of size 
j — 1 from a population whose distribution is simply Fix) truncated on the 
right at jc y . 
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Proof. From the marginal density function of X,.,, in (2.2.2) and the joint 
density function of X i: „ and Xj (1 < / < j < n) in (2.3.2), we have the 
conditional density function of A',.,,, given that A ' j: „ = x jt as 

fr.n( X i\ X j-.„ =X i) = fi.j:n( X ,’ X j)/fi:„( X i) 

( 7 - 1 )! i Fix,.) )''-' 

(i ~ 1 ) !( 7 * 1 ) ! \F( x j) ) 

/ ^(x y ) - /(X t .) 

X \ F(xj) I F(xj)' 

1 < / < 7 , -oo < jt, < Xj. (2.4.4) 

The proof is completed by noting that Fix,)/Fixj) and f(x j )/F(x J ) are the 
cdf and the density function of the population whose distribution is obtained 
by truncating the distribution Fix) on the right at Xj. □ 

By using an argument similar to the one applied in Theorems 2.4.1 and 
2.4.2, we establish in the following theorem that the sequence of order 
statistics from an absolutely continuous population constitute a Markov 
chain. 

Theorem 2.4.3. Let {A 1 ,}, be a sequence of independent random variables 
from an absolutely continuous population with cdf Fix) and density function 
fix), and {A r , : „}, be the corresponding sequence of order statistics. Then this 
sequence of order statistics forms a Markov chain. 

Proof In order to prove this result, we first obtain from (2.2.3) the joint 
density function of A' l .„, X 2 .„ X to be 

/l ,2 j : n i x l’ X 2 i • • • i X j) 

= n\f(x x )fix 2 ) ■ ■ ■ f (\ :,) 

X {/ f ' + 7 ,+ V(Jfi+i ) /(Jf/+ 2) " • /(x M ) d*,- + | d* l - +2 ■ ■ dx „ J 

n\ 

- 7 r-rO - FixiW-fixOfiXi) • • • fix,), 

in - / ) ! 

-oo < x, < x 2 < ■■■ <jr,<oo, (2.4.5) 
and similarly the joint density function of X x ,„, X 2 .„ X i n ,X r j:n 
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( 1 < i < j < n ) to be 

/ 1,2 i,j:n( X l' T 2 , • • • , Xj, Xj) 

= n\f(X i )f(X 2 ) ' ' ' f(Xj)f(Xj) 

rr- 2 

'Xj *x t J x 
C x i+* r x 


f 1 ... j , + 'Y '"7(t, + 1 )/(t, +2 ) • •• f(Xj- l )dx i+l dx i + 2 ■■■ dxj_ 

J Xj J x t J x, 

f " ■ / * 7. ' ""/(-*'>+ 1 ) /(Jfj+ 2 ) • ■ • /(•*,.) ^ + i dx J + 2 ■■■ dx„ 


f / / 


X/(T,)/(T 2 ) • • • /(*,.)/(*;), — 00 <JTj <JT 2 < ••• <X,<Xj<x. 

(2.4.6) 


From Eqs. (2.4.5) and (2.4.6), we obtain the conditional density function of 
Xj. n , given that A", = x x , X 2 .„ = x 2 , . . . , X i:il = x r to be 

fj:n{.Xj\X | ; ,1 _ I I ■ • • 1 Xj . n X t j 

= /l,2 ,-. / :„(t,,T 2 ,...,T,,T j )//| 2 ,-;„( Jf,, jr 2 Jf,.) 

(«-/)! 

(j-i - \)\(n -j)\ 


fF( Xj )-F(Xj)\ J — 1 

1 I-F(xj) 

\"' J f(Xj) 

\ 1 -/=■(*,) / 

\ 1 - 

1 l-f(x,) 


-* < T| < jr 2 < • • • < Xj < Xj < OC. 


The proof is completed simply by noting that this is exactly the same as the 
conditional density function of X j ir given that X ril = x r derived in Eq. 
(2.4.3). □ 


The results presented in Theorems 2.4.1 and 2.4.2 can also be generalized 
to relate the conditional distribution of order statistics (conditioned on two 
order statistics) to the distribution of order statistics from a population whose 
distribution is a doubly truncated form of the original population distribution 
function. This result is presented in the following theorem. 

Theorem 2.4.4. Let X x , X 2 ,..., X n be a random sample from an abso- 
lutely continuous population with cdf F(x) and density function fix), and let 
X i . n <X 2 .„ < ••• < A",,.,, denote the order statistics obtained from this 
sample. Then the conditional distribution of X r „, given that X j:n = x, and 



26 


BASIC DISTRIBUTION THEORY 


X k : n = x k for i < j < k, is the same as the distribution of the O' - i)th order 
statistic in a sample of size k - i - 1 from a population whose distribution 
function is F(x) truncated on the left at jc, and on the right at x k . 

Proof. By adopting a method similar to the one used in deriving the joint 
density function of two order statistics in (2.3.2), we can show that the joint 
density function of X in , X rn , and X k .„ (1 <. i <j < k ^ n) is given by 

fi,j,k:n{ x i> x j > x k ) 

n! 

= (i - l)\(j - i - l)\(k - j - l)l(n - k)\ 

x^*,)}' -1 ^*/) ~ F (•««)}' 1 

~ f (•«/)}* ' *{! ~ F ( x k))"~ k f( x i)f( x j)f( x k)> 

-» < jc . < Xj < x k < oo. (2.4.7) 

From Eqs. (2.3.2) and (2.4.7), we obtain the conditional density function of 
X j:n , given that X i:n = jc, and X k:n = x k , to be 

fj-.n^j^Xj-,, ~ Xj, X k:n — X k ^ 

— fi,),k :n( x i’ X j’ x k)/fi,k :n( X i’ x k) 

_ (k-i- 1)! f F(Xj) - F(x,) 

(j-i-l)!(k-j-l)!\F(x k )-F(x i )j 

( F(x t )-F(x,) f(xj) 

X \F(x k )-F(x i )j F(x k )-F( Xi )’ 

X, < Xj < x k . (2.4.8) 

The result follows immediately from (2.4.8) upon noting that {Hxy) - 
FU,))/{FU*) - FU,)} and /(jc;)/{F(jc*) - F( jc,)} are the cdf and density 
function of the population whose distribution is obtained by truncating the 
distribution F(x) on the left at jc, and on the right at x k . □ 

In addition to all these results, order statistics possess some more interest- 
ing distributional properties if the population distribution is symmetric, say 
about 0. In this case, by using the facts that /(-jc) = fix) and F(-x) - 
1 - F(jc), we observe from Eq. (2.2.2) that 


(2.4.9) 
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similarly, from the joint density function of X i:n and X j:n in (2.3.2) we 
observe that 

{X i:n ,X j:n ) ± (-X n _ j+l[n ,-X n _ i+l:n ). (2.4.10) 


These two results help reduce the amount of computation to be performed 
for the evaluation of moments of order statistics in the case of a symmetric 
population. For example, from (2.4.9) and (2.4.10) we have 


and 


Ml = (-i)V"- , i 


+ I : n 


/ -viWi+my ( OT OT .) 

M' i,j:n V l ) f* , n-j+\,n-i+ l:n* 


In particular, we obtain from these results that 


V'i.n Mn-j+liB* 
i : n - i + 1 , n — / + 1 : n * 

~ Mn-/ + l,n-/+l;n> 


and 


(2.4.11) 


(2.4.12) 


(2.4.13) 

(2.4.14) 

(2.4.15) 


(2.4.16) 


2.5. DISTRIBUTION OF THE MEDIAN, RANGE, 

AND SOME OTHER STATISTICS 

Consider the sample size n to be odd. Then, from Eq. (2.2.2) we have the pdf 
of the sample median X n = X (n+1)/2 . n to be 

/4r) = (i^ 4 / 2 T l ) zlF( ^ <1 ~ f(J)>l<, ~' >/V(j) ^ 

-oo < x < oo. (2.5.1) 

From the pdf of the sample median in (2.5.1), we see at once that it is 
symmetric about 0 if the population distribution is symmetric about 0. We 
can work out the moments of the sample median X n from (2.5.1). For 
example, in the case of the standard uniform population, the pdf of the 
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sample median given in (2.5.1) becomes 
nl 


fo.i u ) 


U <»-|>/ 2 (1 _ 0 < u < 1, (2.5.2) 


{[(/« - l)/2] !} 

from which the wth moment of U„ is obtained as 

, «! [(« - l)/2 + m \ ! 

■ (,+,.)! ((-l)/2). ■ m = '- 2 (2 - 5 ' 3) 

In particular, we have the mean and variance of U n to be 
E(C„)-\ and 

Similarly, in the case of the standard power-function distribution, the pdf 
of the sample median in (2.5.1) becomes 


fxi x ) = 


{[(« - l)/2]!} 2 


vx 


»•[(«+ D/2]- 1(1 _ x i>) < "-l > /2 


0 <* < 1, v > 0, (2.5.4) 
from which the mth moment of X n is obtained as 

T(n + 1) r[(n + l)/2 + m/v] 

£ (*">° r (.!.+,/,) no. + 0/21 - ■ (25 ' 5) 


Suppose the sample size n is even. Then, as defined in the Notations and 
Abbreviations list, the sample median is given by X„ = (* ( „ /2): „ + 
A' (n/2 , + i ,„)/2. In order to derive the distribution of X n in this case, we first 
have from (2.3.2) the joint density function of X (n/lr „ and -Y ( „ /2)+ , ; „ to be 

nl 

f, i/2.»/2 + . *2) = [{n/2 _ 1)!}2 

x {F( t ,)}" /2_I {1 - 

-00 < Ti < x 2 < o°. (2.5.6) 
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From (2.5.6), we obtain the joint density function of X n/2 .„ and X n to be 

2m! 

fx , x ( x \ , x ) = 7 

" {(m/ 2 - l)!} 2 

-F(2x- Xi )}" /2 -'f( X] )f(2x-x t ), 
-oo < jc , <jc<oo. (2.5.7) 

By integrating out jc, in (2.5.7) we derive the pdf of the sample median X n as 
2n ! 

fx ( x ) = 7 

' {(«/ 2 - 1 )!)“ 

x/ { /r ( -^ 1 )}' /_ ‘{1 - F(2x - AT, )}" /2 I /(t i )/(2t — T|) dx | , 

•' — -X 

-oo < jc < oo. (2.5.8) 

The integration to be performed in Eq. (2.5.8) does not assume a manageable 
form in most cases. Yet the cdf of the sample median X n can be written in a 
simpler form from (2.5.8) as 


Fx /i (x ( ,) = P(X n <x t) ) 

= — 2 f X {/=■( jc,)}" /2_ ‘{1 - F( 2x - Jc,)}" 72- ' 

{(m/2 - 1 )>} ZJ -J-~ ' ' 1 V ' n 

x /(t,)/(2t - t,) dx { dx, -oc < jc (| < oo. (2.5.9) 

By employing Fubini’s theorem and changing the order of integration, we 
derive the cdf of X„ as 


Fx,( *o) ~ 


{(m/2-1)!} 


f f"{F( x \)}" /2 'f( x i) 


X / [\ - F(2x - X\)) n/ ~ 'f(2x-x l )dx dx 


w T-"1k./ 3I - fCAir 

- f"{F{ X ,)) n/2 -'{\-F{2x,-x {) ) n/1 f(x { )dx\. (2.5.10) 
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In deriving (2.5.10) we have assumed that the population distribution has an 
infinite support. When the population distribution has a finite support, we 
may still use the expression in (2.5.10), but after fixing the limits of integra- 
tion carefully. We shall illustrate this by considering the standard uniform 
population. In this case, as 1 - F( 2x 0 -r,)s0 whenever jc, ^ 2x 0 - 1, we 
obtain the cdf of X n from (2.5.10) to be, when 0 ^ x 0 ^ 


Fx.( x o) ~ 


ft! 

(ft /2 - 1 ) !(ft/ 2 ) ! 
X 


[/ 0 IOjt > /2 ~ ,(1 - x i'> n/2dx i ~ f*°*i /2 - i (l+x l - 2x 0 ) n/2 dx l 

-'4w + ‘) - f) (1 

X / ° x i /2 ~ l (x 0 -x l )‘dx l 
J o 

0^x o ^i, (2.5.11) 


and when j <, ^ 1 

n! 


Fx(x 0 ) - 


(ft/2 - l)!(ft/2)! 
X 


[ f V 2-1 a - x0 n/2 - P X n / 2 ~'(l + x, - 2x 0 ) n/2 dx, 

j/0 J 2x 0 - 1 

/« « \ «! 

= Mi’ 2 +l )~ («/2 - l)!(«/2)! 

x E ( n/2 . 1 )(2jt 0 - l) n/2 ' ,_, (l -x 0 ) n/2+i+l 

i=0 ' 1 > 

X pt n/2+i dt 

X(2x 0 - i)" /2 - , -'(i -* 0 ) n/2+/+1 , X -zx Q z\. (2.5.12) 


In the foregoing formulas, / Io (n/2, n/2 + 1) denotes an incomplete beta 
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function as used in Eq. (2.2.15). In particular, when x 0 = we obtain from 
Eqs. (2.5.11) and (2.5.12) that 



Next, we shall describe the derivation of the distribution of the sample 
range W„ = X n:n - X X:n . From the joint density function of X X:n and X n , n 
in (2.3.7), we have the joint density function of X Xn and W n as 

fx x „,w„( x i, w ) = n(n ~ 1){F(*, + w) - F( x x )}"~ 2 f(x x )f(x x + w), 

-oo <x<oo ; 0<iv<oo. (2.5.14) 

By integrating out x, in (2.5.14) we derive the pdf of the sample range W n as 

fw„( w ) = ~ l )f { F ( x i + *) ~ F{ x x )r 2 f{ Xl )f{ Xx + w) dx x , 

0 < w < oo. (2.5.15) 

Even though the integration to be carried out in (2.5.15) does not assume a 
manageable form in many cases, the cdf of W n does take on a simpler form 
and may be derived as 


F w„( w o) = <; w 0 ) 

= n(n - 1 )f ° f {F(x, + w) - F(x l )} n ~ 2 f(x l )f(x l + w)dx x dw 

J Q J — 00 


= nf f(x x ) (n - l)f °{F(x x + w) - F(x)} n 2 f(x , + w)dw 

J — oo 


dx , 


= «/_ {F(x, + w 0 ) - F(x x )) n l f(x x )dx x , 0 < w 0 < oo. (2.5.16) 


It should be mentioned here that the expressions of the density function and 
cdf of W n derived in (2.5.15) and (2.5.16) are for the case when the 
population distribution has an infinite support, and may still be used for 
the case when the population distribution has a finite support by changing 
the limits of integration appropriately. For example, for the standard uniform 
distribution, since f(x x + w) == 0 when x x + w > 1, we obtain from Eq. 
(2.5.15) the pdf of the sample range W n as 


fw„(w) = n(n - 1) f l 2 dx x 
J o 

= n(n - l)w n ~ 2 (l - w), 0<w<l. (2.5.17) 
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From Eq. (2.5.17), we obtain the cdf of the sample range W n as 

F w (w n ) = ( "n(n - \)w"~ 2 (\ - w) dw 
" ■'o 

= nw(l~ l - (n - IX, 0 < w {) < 1. (2.5.18) 

Also, by realizing that F(x t + w 0 ) = 1 when jc, > 1 - w () , we obtain the cdf 
of W n from Eq. (2.5.16) as 

M vv ») = / 1 dxt+nf (1 -x l )'^'dx l 

= nw ( ” ~ 1 ( 1 - w„) + w,", 0 < w 0 < 1, 

which is exactly the same as the expression derived in (2.5.18). It is of interest 
to note from (2.5.17) and (2.5.18) that the sample range W n from a standard 
uniform population has a Beta(n - 1,2) distribution. 

The results for the sample range discussed above can be generalized to the 
spacing W ( j: „ = X j n - X i:n , 1 < i <j < n. It may be noted that the /th 
quasirange W i: „ = X n _ i+l:n - X i n is a special case of and hence a 

spacing is sometimes called a generalized quasirange. In order to derive the 
distribution of W i y .„, we first obtain the joint density function of X i:n and 
Wj j. n from Eq. (2.3.2) to be 

t , \ n\ 

= (/_ 1 ) !( y — / — 1 )\( n - j ) ! 

x { /r (^,)}'" l { /r (' t , + w ) ~ 

X{1 -F(x i + w)} n - i f(x i )f(x i + w), 

-ao <x , < °°, 0 < w < oo. (2.5.19) 

By integrating out jc, from (2.5.19), we derive the pdf of W t t . n as 

n\ 

= (/- 1 ) ! ( y - / - 1 )!(m — j ) ! 

xf [F(x i )}-'{F(x i + w)-F(x l )r'-' 

J — 00 

X{1 - F(x, + w)Y~ , f(x i )f(x, + W)dx„ 


0 < w < oo. (2.5.20) 
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For the standard uniform distribution, for example, we obtain the pdf of 
Wj j. n from (2.5.20) to be 




w-' ' 1 

(/- l)!(/-i- 1 ) !(« ~j)'. 

X f l xj~'(l -w-Xj)" 1 dXj 
•'ll 


n ! 


U - i -!)!(« ~j + O'- 




0 < w < 1 . 
(2.5.21) 


We thus observe that W, has a Beta(/ - /', n - j + i + 1) distribution 
which depends only on j - i and not on i and j individually. Further, we 
note from Eq. (2.5.21) that for the standard uniform distribution the spacing 
Wj j is distributed exactly same as the (/ - / )th order statistic in a sample 
of size n from the standard uniform distribution. Many more interesting 
distributional properties of this nature for order statistics from uniform and 
some other populations are discussed in Chapter 4. 

Next we shall present the derivation of the distribution of the sample 
midrange V„ = (A",.,, + X n . n )/2. From the joint density function of X l:n 
and X ti:n in (2.3.7), we have the joint density function of Aj.„ and V n as 

fx y H (x ltl ')^2n(n - l){F(2r - *,) - F( t, )}"‘ 2 /( )/(2c - a,), 

-ac <T, < (' < X. (2.5.22) 

By integrating out t, in (2.5.22) we derive the pdf of the sample midrange V n 
as 


U H (r)=2n(n- 1)/' {F( 2r-x,) - F(x,))"~ 2 f(x,)f(2r - x,) dx„ 

-cc < i < x. (2.5.23) 

For example, for the standard uniform population, we obtain from Eq. 
(2.5.23) the pdf of V n to be, when 0 < r < 

/c„(' ) = 2 n(n - l)jf'(2t- - 2 a ,)" 2 dx t 
= 2"- l m " \ 0 < i' < 4 , 


(2.5.24) 
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and when v < 1, 


f v (v) = 2n(n - 1) f (2v - 2x,)" 2 dx, 

" hv - 1 

= 2" -1 /i(l - v) a ~ l , (2.5.25) 

From Eqs. (2.5.24) and (2.5.25), we derive the cdf of the sample midrange V„ 
for the standard uniform distribution as 

Fy m (v o) = P (V n * v 0 ) = 2"-'uJ, if 0 < u 0 <; £ 

= 1 - 2"-‘(l - w 0 )". if \ <■ v 0 <Z 1. (2.5.26) 


From Eq. (2.5.23), on the other hand, we may write in general the 
distribution function of the sample midrange V n as 


F y (v 0 ) = 2 n{n - 1) f ° / (F(2v - x,) - F(x,)} n 2 f(x,)f( 2v - x,) dx, 

" ■'-oo-'-oo 

= nj V °f(x i) 2(n - l)f°{F(2v - x,) - F(x,)}"- 2 /(2i; -x,)do 

= n p (F(2i; 0 - x,) - Ffx,)}" - ’/^!) dx„ -» < i> 0 < 

J — 00 


di> 


dx, 


(2.5.27) 


For the standard uniform population, the cdf of the sample midrange V„ 
obtained from (2.5.27) is identical to the one derived in Eq. (2.5.26). 

Similar formulas can be derived for the density function and the dis- 
tribution function of the general quasimidrange V iJ n = (X i:n + X j:n )/2, 
1 <i <j <n, which of course will include the ith quasimidrange V i:n = 
(X in + X n _ i+l:n )/2 as a special case. 


EXERCISES 

1. Let X v X 2 , and X 3 be i.i.d. Exp(0) random variables with pdf 

/(x) = \e~ x/ \ x ;> 0, 0 > 0. 
d 

(a) Determine the distribution of X I:3 . 

(b) Determine the distribution of A3.3. 

(c) Determine the distribution of the range X 3:3 - X J:3 . 
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2. Let X t and X 2 be i.i.d. Pareto ( v ) random variables with pdf 

f{x) = vx~ v ~ 1 , x > 1 , v > 0 . 

(a) Discuss the distribution of the following random variables: 

(0 * 1 : 2 . ('0 -^ 2:2 — : 2 > ( *'*) -^ 2 : 2 /^ 1 : 2 • 

(b) Can you predict analogous results for order statistics based on 
samples of size larger than 2? 

3. Let X { and X 2 be i.i.d. random variables from a population with pdf 

/(jc) = ^sinjc, 0 < x < rr 

= 0, otherwise. 

(a) Determine the distribution of X 2 . 2 . 

(b) What about X n n based on a sample of size n? 

4. Let X x and X 2 be i.i.d. Exp(0) random variables. Define 

Z =X ]:2 /(X 1 +X 2 ) and W = X l /(X i + X 2 ). 

Determine the distributions of Z and W. 

5. Now try Exercise 4 with n = 3. In other words, discuss the distributions 
of 


(Z„Z 2 ) 


I *.:3 * 2:3 \ 


and {W X ,W 2 ) 


\LUxr Li,x t J- 


6. Let X lt X 2 , X 3 , and X 4 be i.i.d. Exp(l) random variables. Then, find 
P(3<* 4:4 ) andPU 3;4 >2). 

7. Let X,, X 2 , and X 3 be i.i.d. random variables from a triangular distribu- 
tion with pdf 


f(x) = 2x, 0 < x < 1 

= 0, otherwise. 

Calculate the probability that the smallest of these Z/s exceeds the 
median of the distribution. 
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8. For the Pareto distribution with density function 

f(x) = ux~ r ~ l , x>l, v > 0, 

show that 


„„) = + 1) r(n -i+l -m/v) 

M,:n r(n-/+l) T(n + 1 - m/v) 

which exists for v > m/(n - i + l). Similarly, show that 


= T(« + l) r (n-j+l-m/u) 

T(n -; + 1) r(n - / + 1 - mjv) 

r[n - / + 1 - (m, + mj)/v] 
r[n + 1 - ( m , + nij)/v ] 

which exists for 


v > max 


m j 


n -j + 1 


m t + m J \ 
«-/+!/' 


(Huang, 1975) 


9. For the standard exponential distribution with density function 

f(x)=e~\ x>0, 

show that 

Mi:„ = l/« and cr, | .„ = 1/n 2 . 

What can you say about the distribution of A',.,,? More generally, can 
you show that 

- E 1A and o = E 1A 2? 

f = n - i + I r = n - / + 1 

(See Section 4.6 for details.) 


10. Let A" be a random variable with density function 

1 ir 

f(x) = -cos(x), Ul < -• 
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This is referred to as the sine distribution, since U = sin X is uniformly 
distributed over [-1, I], Show in this case that 



Can you derive an expression for rr„ 


(Burrows, 1986) 


11. Consider the triangular distribution with density function 

f{x) = 1 - \x\, -1 <* < 1, 

and cdf 

F(x) = t( 1 + x)\ — 1 < jc < 0 

= 1 — 3(1 -*)\ 0 <* < 1 . 
Derive an explicit expression for n i n and <r ( , . „ in this case. 

12. Consider the Weibull population with density function 

f{x) = e-'* 8x*~\ x>(), 8 > 0. 

In this case, show that for i = 1,2,..., // 



x r<-iW‘ 7 ')/(» 

r = 0 V 11 

Can you similarly derive an expression for the product moment p., , .„? 

(Lieblein, 1955; Balakrishnan and Cohen, 1991) 

13. For the logistic distribution with density function 

/( X) - e~'/( 1 + C~ x ) 2 , -3C<A'<x, 

show that the moment-generating function of X j . n is 
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Then, show that for / = 1, 2 n, 


Hi:n = *(0 ~ H n - 1 + !) and = ^'(0 + ^'( n “ * + 1)> 

where t/»(z) = d/dz log T(z) = T'(z)/T(z) is the digamma (or psi) func- 
tion and i//'(z) is the derivative of i/t (z) known as the trigamma function. 
(See Section 4.8 for details.) 

(Bimbaum and Dudman, 1963; Gupta and Shah, 196S; 

Balakrishnan, 1992) 


14. Consider a gamma population with density function 


/(*)-— x^O, 


where p is a positive integer. Show in this case that 
2 <«-iXp-D 


,.(m) — 

Pl:n 


r , , E a r (p,n-l)T(m + p + r)/n^ +r - 1 , 
np) ,-o 


where a r (p, N ) is the coefficient of x r in the expansion of 


fp-i 
/- 0 


, N 


Show then that for 2 <,i <, n, 

»<«> = - V ( - 1 )'(*' - 1 

M,:n (/-l)!(«-«)!r(p) r _o ' r 

(n—i + rXp— 1) r(m+p + i) 

Can you suggest a recursive process for the computation of the coeffi- 
cients a r (p, N)? Can you similarly derive an expression for the product 
moment for 1 <.i <j <.nl 

(Gupta, 1960, 1962; Balakrishnan and Cohen, 1991) 

15. A distribution Fix) is said to be an IFR (increasing failure rate) [DFR 
(decreasing failure rate)] distribution if the conditional survival probabil- 
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ity is a decreasing (increasing) function of age, viz., 


F(t|x) 


1 - F(x + t) 
1 -*■(*) 


is decreasing (increasing) in 0 < x < °o for every t > 0, or, equivalently, 
when the failure or hazard rate 


h(x) 


1 Fix + t) 

lim — — - 

/-»o t 1 - F(x) 

lim — [ 1 - 
(-.0 t 


F{x) 


1 - F(x + t) 

1 -f(*) 


is increasing (decreasing) in x > 0 


[h(x) = f(x)/( 1 - Fix)) whenever the density f(x) exists]. Prove that if 
F is IFR, so is F in . 

(Barlow and Proschan, 1981) 


16. A distribution Fix) is said to be an IFRA (increasing failure rate 
average) [DFRA (decreasing failure rate average)] distribution if 

1 

- — Iog{ 1 - F(x)} is increasing (decreasing) in x > 0. 

It should be noted that — Iog{ 1 - F{x)} represents the cumulative failure 
rate fghlu) du when the failure rate h{u) exists. Prove that if F is IFRA, 
so is F hn . 

(Barlow and Proschan, 1981) 


17. If is IFR, then show that F i+ is also IFR for i = l,2,...,n - 1. 
Similarly, show that F j _ l n is DFR whenever Fj. n is DFR for i = 
2,3,..., n. 

(Takahasi, 1988) 

18. If F,.„ is IFR, then show that F i . n _ l and F i+l:n + l are also IFR (in 
addition to F i+l:n ). Similarly, show that if F). n is DFR then F i _ l;n _ l 
and F i:n+l are also DFR (in addition to F i _ l:n ). 

(Nagaraja, 1990) 

19. Show that the results given in Exercises 17 and 18 continue to hold when 
IFR and DFR are replaced by IFRA and DFRA, respectively. 

(Nagaraja, 1990) 

20. A distribution F(x) is said to be a NBU (new better than used) [NWU 
(new worse than used)] distribution if the conditional survival probability 
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{1 - Fix + y))/{l - F(jt)} of an unit of age x is smaller (larger) than 
the corresponding survival probability 1 - F(y) of a new unit; that is, if 

1 - F(x + y) < (>){1 - F(jt)}{l - F(y)}, forjr^O, y > 0. 

Show then that the results given in Exercises 17 and 18 continue to hold 
when IFR and DFR are replaced by NBU and NWU, respectively. 

(Nagaraja, 1990) 

21 . For the standard uniform population with density function 

f(u) = 1, 0 < m < 1, 

prove that the random variables U i n /Uj. n and Uj.„, for 1 £ i < j < n, 
are statistically independent. What can you say about the distributions of 
these two random variables? (See Section 4.7 for more details.) 

22 . For the standard exponential population with density function 

f{x) = e~ x , 0 < x < <*>, 

prove that the random variables nX x .„ and in - IXX 2 .„ - X x .„) are 
statistically independent. What can you say about the distributions of 
these two random variables? (See Section 4.6 for more details.) 

23 . For r = 1,2 ,...,« - 1, show that 

Fr. n ix)=F r+l:n ix) + ("){F(x)} r {l -Fix))"-' 
and 

F r -.nix)=F r , n _ x ix) + (”:{){F(*)f{l -F( J f)}"“ r . 

(For more such recurrence relations, refer to Chapter 5.) 

(David and Shu, 1978) 

24 . Life rapidly becomes complicated when we do not require Xfs to be 
identically distributed. For example, consider X x , X 2 , and X 3 to be 
independent random variables with X, having an Expffy) distribution 
(/ = 1,2,3); assume that 0,’s are distinct. 

(a) Determine the density function of X l 3 . 

(b) Determine the density function of X 2:J . 

(c) Are X x 3 and X 2:3 -X l:3 independent? 

(d) Are X 2:3 - X, :} and X 3:} - X 2:3 independent? 
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Discrete Order Statistics 


3.1. INTRODUCTION 

The basic distribution theory of order statistics developed so far has assumed 
that the random variables constituting the random sample are absolutely 
continuous. In this chapter we explore the discrete case, where the basic 
assumption is that A',,..., A',, are i.i.d. random variables having a common 
discrete cdf F. We present formulas for the probability mass function (pmf) 
of a single order statistic and the joint pmf of two or more order statistics. 
We study the dependence structure of order statistics in random samples 
from discrete populations and discover that, in contrast to the continuous 
case, the A',.„’s do not form a Markov chain. We also derive an expression 
for the pmf of the sample range, W n . We discuss two examples in detail, 
namely, when the parent is discrete uniform and when it is geometric. Order 
statistics from the binomial and Poisson distributions will be studied in 
Chapter 4. In the last section we discuss the distribution theory of order 
statistics when we have a simple random sample drawn without replacement 
from a finite population consisting of distinct values. In such samples, while 
Ays are identically distributed, they are no longer independent. 

Several of the asymptotic results to be developed in Chapter 8 are 
applicable to discrete populations also. These will be pointed out at appro- 
priate places in that chapter. Further information on the properties of order 
statistics from discrete distributions may be found in the discussion article, 
Nagaraja (1992), which also contains a survey of the literature. 


3.2. SINGLE ORDER STATISTIC 

We will now obtain three expressions for the pmf of the r th order statistic. 
The first two are based on the expressions for the cdf F in (x) obtained in 
Section 2.2. The last one is based on a multinomial argument. We also obtain 
expressions for the first two moments of A',.,,. 
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Approach 1 (Binomial Sum) 

Recall (2.2.13), which yields an expression for F i:n ix). In the discrete case, 
for each possible value x of X in , we have 

f, ,.(*)- -)■ f 3 - 2 - 1 ) 

Consequently, we may write 

f„M - £ ("){[F<^)]'[1 - F(*)]-' - [F(i -)]'[1 - F(x 

r»i 

(3.2.2) 

One can similarly use the negative binomial sum form for F i:n ix) discussed 
in Chapter 2 to obtain another representation for f i:n ix). 

Approach 2 (Beta Integral Form) 

While the preceding expression is better for computational purposes, an 
expression for f i:n ix) in the form of an integral is useful for studying the 
dependence structure of discrete order statistics. It makes use of the form of 
F i:n i x) given in (2.2.15) and (3.2.1). In other words, 


where 


f i:n (x) = c(«»/ m) «‘- , (i -uy-Uu, 

J F(x - ) 


(3.2.3) 


C(i;n) 


n\ 

(i - 1) !( n -«’)!’ 


(3.2.4) 


Approach 3 (Multinomial Argument) 

In the absolutely continuous case, an argument involving multinomial trials 
was used to obtain the pdf of X i:n given by (2.2.2). That idea can also be 
used here. But the final expression for the pmf of X i:n thus obtained is more 
complicated. This is precisely because the chance of ties is nonzero. 

With each observation X we can associate a multinomial trial with three 
outcomes {X < jc), {X = x), and {X > x), with corresponding probabilities 
Fix - ), fix), and 1 - Fix), respectively. The event {X i:n =x) can be 
realized in iin - i + 1) distinct and mutually exclusive ways as follows: 
ii — 1 — r) observations are less than x, in — i — s) observations exceed x, 

and the rest equal x, where r = 0, 1 / - 1 and s = 0, 1 n - i. Thus, 

we obtain 


Unix) 


r _o*-o (« - 1 - OK" - + '■+!)! 


. (3.2.5) 



SINGLE ORDER STATISTIC 


43 


Example 3.1 (discrete uniform distribution). Let the population random 
variable X be discrete uniform with support 5 = {1, 2, ... , AO. We then 
write, X is Discrete Uniform [1, N], Note that its pmf is given by fix) = 1 /N, 
and its cdf is Fix ) = x/N, for x e 5. Consequently, the cdf of the ith order 
statistic is given by 


Fi.ni*) = 



x e 5. 


One can use the tables for the cdf of the binomial distribution directly for 
selected x and N. For example, when N = 10, every x in S can be expressed 
as x = lOp, p - O.lCO.Dl.O. Thus, for x e 5, 

P i n {x) = E(?)/> r (l -P) n_r . 

r*»i 

which can be read from the binomial tables, and f in ix) can be obtained 
using (3.2.1). 


Moments of X i:n 

As pointed out in Chapter 1, we can use the transformation X i n = F~\U in ) 
to obtain the moments of X in . For example, we can express the mean of 
Xi n as 

fi i:n = C(i; n) / 1 F -1 (u)u ,-1 (l - u)"~‘ du, 

where C(i; n) is given by (3.2.4). However, since F~ l iu) does not have a nice 
form for most of the discrete (as well as absolutely continuous) distributions, 
this approach is often impractical. When the support 5 is a subset of 
nonnegative integers, which is the case with several standard discrete distri- 
butions, one can use the cdf F i:n ix) directly to obtain the moments of X i n . 

Theorem 3.2.1. Let S, the support of the distribution, be a subset of 
nonnegative integers. Then 

XI 

F i:n = £{1 -^(jc)}, (3.2.6) 

and 

oo 

tf n = 2T.x{l-F i:n (x))+p i:a , (3.2.7) 

*-o 

whenever the moment on the left-hand side is assumed to exist. 
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Proof. First let us note that if ti in exists, kP(X i:n > k) -» 0 as k -* <». 
(Why?) Now consider 

E *fU*) = E *{*>(*,:,. > x - 1) - P(X hH > X)} 

Jt-0 x=0 

= *E{(* + 1) -x}P(^ (:n >x) - kP(X i;n > k). 

x-0 

On letting k -» », we obtain 

/*,:■ = Jim EP(X i:n >x) = £ {1 -F l:il (x)}, 

k ~°° x-0 x-0 

which establishes (3.2.6). To prove (3.2.7), we begin by noting that 
E * 2 / ( :„(*) = *E {(X + l) 2 - X 2 }P(2T i;n > x) - * 2 P(* j;n > *) 

jt-0 Jt-0 

= 2 k ZxP(X i:m >x) + LP(X i:n >x) - k 2 P(X i .„ > k). 

jt-0 jt-0 

As k -» », the last term on the right-hand side approaches 0, while the 
middle term approaches fi iin . Thus, we obtain (3.2.7). □ 

In general, these moments are not easy to evaluate analytically. Some- 
times, the moments of sample extremes are tractable. Let us see what 
happens in the case of discrete uniform distribution. 


Example 3.1 (continued). When X is a Discrete Uniform [1, N] random 
variable, in the case of the sample maximum, (3.2.6) yields 


Pn:n = L(l-U')} 

Jt-0 

= eV -(*/*)"} 


(3.2.8) 


The sum on the right-hand side of (3.2.8) can be evaluated easily. Abramowitz 
and Stegun (1965, pp. 813-817) have tabulated it for several n and N values. 
For n up to 10, algebraic expressions for the sum are available in Beyer 
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(1991, p. 414). Using these, we can conclude, for example, that 

M 2:2 = (4Af- 1)(N + l)/6/V and M 3:3 = (3N 2 + IN - 1)/4N. 
Further, from (3.2.7), 


/*«■« = 2 £ *{1 * (x//V)"} + /i n;n 

JC= 1 

x = l 

= N(/V - 1) + 2N(fi n+l:n+l — N) + fi n n , 

and hence, 

<*l.n = 2 Nfi n + l:n + \ - N - N 2 + ti n n - ti 2 n:n . 
When n = 2, using the values of /r 2: 2 and m 3:3 , we obtain 


(T 


2 _ 
2:2 ~ 


(2N 2 + 1 )(N 2 - 1) 
36 W 2 


(3.2.9) 


3 3. JOINT PROBABILITY MASS FUNCTION 


We now obtain the joint pmf of k order statistics represented by 
Af, |;n , . . . , X ik . n , where 1 < i, < i 2 < • • • < i k < rt. A convenient form for 
the pmf can be presented as an integral. For this we start with an integral 
form for the joint pmf of all order statistics and sum with respect to the 
remaining order statistics. Here the challenge is in keeping track of the 
number of ties among the observed values. 

Let x x < x 2 < ■ ■ ■ < x n be such that x x = ■ ■ • = x r < x r +1 = - • • = 
x r 2 < •• = 1 = • =x r m > 1 ^r x <r 2 < '■■■ '< r m = n, for 

some m, 1 < m < n. Then with r 0 = 0, 


where 


/l, 2 «:»(jf|. ■■■,*„) -n\Yl 

S =* 1 


(r s -r s _ x )\ 



(3.3.1) 


(3.3.2) 


D = {(«, m„): m, <u 2 < 


<u n ,F(x r -) zu,<F(x ri ), 

1 <s <m,r s _ x + 1 <t </•,}. (3.3.3) 
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When k < n, for x (i ^x i2 <, • • • <, x it , 






• *i t ) = E/i, 


2 ,... 


„:„(*! *„). (3-3.4) 


where E stands for the sum over all x r ’s other than x, x it subject to 

x, ^ x 2 £ ••• ^ x„. On using the representation (3.3.2) for /, j2 ,..., n:n . the 
right-hand side of (3.3.4) can be expressed as n\L,j Di du t ••• du n , where 
D,’s are disjoint and are of the form of D given by (3.3.3). These D,’s 
correspond to distinct configurations of ties among x r ’s. Further, 

UA = {(“i •• ^u n ,F(x ir -) <,u ir <,F(x ir ),\ <.r <,k). 

(3.3.5) 


Thus, we obtain 

/;„, 2 , >*<2 *.*) =n 'j UD du i d “2 •• du n , (3.3.6) 

where U D, is given by (3.3.5). On integrating out w r ’s for r other than 
i„ i 2 i* we get, with /„ s 0 and «„ = 0, 

f du t ••• = ((n - i k )\ fl (i r - i r _, - l)l\ 

•'UD, V r= 1 f 

X / b { FI (“/, " d “i, • • • du i k ' 

where B is the ^-dimensional space given by 

« = {(«,, u it ):u lt £u h Z •• <,u it , F{x,~) Zu r zF(x r ), 

r = iuh «*}• (3-3.7) 


This discussion can be summarized as follows. 

Theorem 33.1. For 1 < i, < i 2 < ■ • • < i k ^ n, the joint pmf of 
X ik , m is given by 


/i'i,i 2 i* :n(-*i,» x i 2 ’ ‘ ‘ > X i k ) 

= ^'(* 1 > *2 *0 


x /J £(«,. - -«*V < 3 - 3 


8 ) 
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where i 0 = 0, u 0 = 0, 


C{i u ...,i k ;n) = /i» 




r= 1 


1)! , (3.3.9) 


and B is given by (3.3.7). 


The above result expresses the joint pmf of k order statistics as a &-fold 
integral. The region of integration, B, can be expressed as a product space of 
subspaces of lower dimensions as long as the order statistics are not tied. For 
example, in the case of no ties, B is a ^-dimensional rectangle. However, 
even in this case a simple expression for the joint pmf is not available unless 
the order statistics of interest are consecutive ones. 

Besides providing a compact form for the joint pmf, the integral expres- 
sion in Theorem 3.3.1 becomes a handy tool in obtaining simplified expres- 
sions for the product moments of order statistics when the support of the 
distribution consists of nonnegative integers. For example, it follows from 
Balakrishnan (1986) that 


m, = /4 2) „ + (”) E *[{*■(*)}'- iH*- i)}'] E 0 


(3.3.10) 


for 1 < i < n - 1. He also gives a similar expression for fi i ,j. n for j > i + 1. 

The relation in (3.3.10) connects the product moment with the second 
moment of a single order statistic, which can be computed using (3.2.7). The 
sum on the right-hand side can be explicitly evaluated for the geometric 
distribution. (See Exercise 17.) 

Theorem 3.3.1 is also helpful in establishing the non-Markovian structure 
of discrete order statistics. This will be discussed in the next section. 

One can also use the multinomial argument to obtain the joint pmf of 
Xj . ,,Xj :n . However, it becomes messier with increasing k and will 
involve multiple sums. We have seen from (3.2.5) that even for a single order 
statistic, the form is not pleasant. In Exercise 6 you will see how bad it gets, 
even for k = 2! 

Another approach to find the joint pmf of two order statistics is to use 
differencing of their joint cdf. That is, use the representation 

fi,j:n(Xl>Xj) = Fi.j:n(Xi,Xj) ~ F UlH (Xi ~ , Xj) 

-F, J: „(x„Xj-) +F iJ:n (x i - ,Xj-), (3.3.11) 

where the expression for F, y;n is the same as in the absolutely continuous 
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case, and is given in (2.3.9). Instead, if we use the representation in (2.3.11) 
for the resulting expression for the joint pmf f iJ:n will be of the form 
given in Theorem 3.3.1. (Can you verify this?) 


3.4. DEPENDENCE STRUCTURE 

While order statistics from an absolutely continuous cdf exhibit Markovian 
dependence, such is not the case in general for discrete order statistics. To 
lay the groundwork, we begin with the following lemma, whose proof is 
developed in Exercise 8. 

Lemma 3.4.1. For 0 <. a < b <. \ and positive integers r and s with 
r < s, 

(b-a)f u r (l - u) s ' du < f b u r du f b (l - u) s r du. (3.4.1) 

J a J a J a 

We are now ready to prove the result which establishes the fact that the 
order statistics from a discrete distribution with at least three points in its 
support fail to form a Markov chain. 

Theorem 3.4.1. For 1 < i < n, 

P(X i+i:n =z\X i:n =y, X t _ X:n =*) < P(X i+Un =z\X i;n =y), (3.4.2) 
where x < y < z are elements of 5, the support of the parent distribution. 

Proof. Since x <y <z, from Theorem 3.3.1, we can write 

= C(i - l,i, i + 1; n) f F(x) utfdHi-i du i 

J F(x-) J F(y-) 

X f f(z> (1 -u l+1 )”--'du i+1 (3.4.3) 

and 


fi-i i.n(x,y) = C(i - l,i;n)f x u^du^j y (1-n,)" ' du t . 

J F(x~) J F(y - ) 

(3.4.4) 
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Thus, for jc < y < z, it follows from (3.4.3) and (3.4.4) that 


P( X i+l.n= z\Xi:n = y> = *) 


= (n - i){F(y) - F(y -)} 


/^j(l 


, (3.4.5) 


since (3.3.9) implies that {C(i - 1, i, i 4- 1; n)/C(i - 1, i; n)} = (n - i). Fur- 
ther, 


F(X i+l:n — z\X i:n = y) = 


fi.niy) 


du t - u^)"-'- 1 d« i+1 

= (rt - 0 — 


/£;>'- ‘(1 - 


(3.4.6) 


from (3.2.3) and (3.4.4). On using (3.4.1) in (3.4.6), it follows that 


n x i + un=z \X l:n =y) 

{F(y)-F(y-)} CFU) 


>(n-i) 


/&”>(1 ~ «<) du,'FU-) 


[ M \ n — i — I , 

I (!-«, + 1) <*«, + ! 


= P(Z,. + i :n = z|Z,, n =y,Z|_i ;n =j t ) 
from (3.4.5). Thus, we have shown that (3.4.2) holds. 


□ 


The inequality in (3.4.2) has two messages. First, it shows that as long as 5 
has at least three points, the X, . „’s do not form a Markov chain. But when 5 
has one or two points, such a situation does not arise. In these cases, it can 
be shown that X i:n ’s possess Markovian structure. (See Exercise 9.) Second, 
(3.4.2) says that whenever x < y < z, the conditional probability assigned to 
the event {A r I+]:n =z) given {X i:n =y, X i _ l:n -x) is smaller than the 
probability assigned to the same event given {X t .„ = y}. A natural question 
would be about the behavior of the conditional probability when we have 
further information about the past (about several previous order statistics). 
More generally one can look at the behavior of the conditional probability 
F P(X j + 1 ; n Xj + 1 , . . . , X j :n x j \X t :n Xj, . . . , Xfc :n — Xfc ), where 1 £ 
k < i < j <, n. When i, j and n are held fixed, P* turns out to be a function 
of two variables v x and v 2 , where v { is the number of x,’s with r < i that are 
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tied with and v 2 represents the number of je r ’s with r > i that are tied 
with jc ( . A detailed study of the behavior of P* may be found in Nagaraja 
(1986b). 

Even though X i:n ’s do not form a Markov chain, by expanding the state 
space into two dimensions, one can obtain a Markov process. Ruschendorf 
(1985) shows that the bivariate sequence (X i:n , Af,) forms a Markov sequence 
where Af, is the number of X k:n ’s with k < i that are tied with X i n . 
Further, conditioned on appropriate events, the X i:n ’s exhibit Markov prop- 
erty (Nagaraja, 1986a). For example, conditioned on the event that X t ’s are 
all distinct, the order statistics form a Markov chain (see Exercise 11). 


3.5. DISTRIBUTION OF THE RANGE 

Let us start with the pmf of the spacing W- t j :n =X j n - X i n . On using 
Theorem 3.3.1, we can write 


= L P(X i: n = X>X j:n =X + W) 


-ca.r.n) z /"'» / FI ' + '> 


=5 J Fix-yFix + w-) 

Ui <Uj 


(3.5.1) 


Substantial simplification of the expression in (3.5.1) is possible when i = 1 
and j = n, that is, in the case of the sample range W n . We then have 


P(K = w) -C(l,n;n) £ / Wx> f F(x+w) (u„ - u^" 2 du n du l 

xeS J F(x-YF(x + w~) 


“i<“„ 


Thus, the pmf of W n is given by 


P(W„ = 0)=n(n 



“i<«» 


-u,) n 2 du n du l 


= E{F(*)-F(*-)r 


= L {/(*)}". 


(3.5.2) 
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and, for w > 0, 

P(K = w) = n{n - 1) E j F(X) f Kx+w) (u „ - du n du t 

x < eS J F(x-) J F(x + w - ) 

= E {[^(* + vv) - F(x -)]" - [F(* + w) - jc)] " 

xeS 

-[F(* + w-) - F(x -)]" + [/■(* +w-) - ^(jc)]"}. 

(3.5.3) 

Expressions (3.5.2) and (3.5.3) can also be obtained without using the integral 
expression from Theorem 3.3.1. One can also use a multinomial argument to 
obtain an alternative expression for the pmf of W n . The resulting expression 
may be found in Exercise 13. 

Example 3.2 (discrete uniform distribution). When X is a Discrete 
Uniform [1, N] random variable, the expressions in (3.5.2) and (3.5.3) can be 
further simplified. We then have 


P(K = o) 



l 

N n ~ l 


and 


P(W„ = w) 


N-w 

E 


x=\ 


f(x + w 


1 x + w 

- X \ 

[[ N 

N J 

\ N 

N) 


(x + w-1 X - 1 (X + W - 1 * 

l N N~ ] + { N iv ) / 

= E + 1)" - 2w n + (w - 1)"} 

x=l iv 

(N - w) 

= — {(* + 1) - 2w" + (w - l) n }, 

w = 1 ,..., A - 1 . 


Using the above pmf, one can determine the moments of W. For example, 
when n = 2, E(W 2 ) = (N 2 - l)/3 N and £(1K 2 2 ) = (N 2 - l)/6. Thus, we 
obtain a- 2 2 = varOF 2 XA^ 2 - 1XA 2 + 2)/187V 2 . 


3.6. GEOMETRIC ORDER STATISTICS 

As we have already seen, in general, the distribution theory for order 
statistics is complex when the parent distribution is discrete. However, order 
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statistics from the geometric distribution exhibit some interesting properties. 
Even though the geometric distribution possesses several properties (like lack 
of memory) of the exponential distribution, their order statistics differ in 
their dependence structure. But there are marked similarities. We discuss 
them in Section 4.6, where representations for the exponential and geometric 
order statistics, given, respectively, by (4.6.19) and (4.6.20), demonstrate the 
closeness in their dependence structure. 

Now, let us explore the properties of geometric order statistics. We say X 
is a Geometric^ p) random variable if it has the pmf 

f(x)=q x p, x = 0,1,2,..., 
where 0 < p < 1 and q = 1 - p. 

The distribution of the sample minimum from a geometric distribution is 
given by 


P(2f 1;n >x) = l-F 1;n (x) = {l-F(x)} n 

_ ^n(x + l), jc = 0,1,2,..., 

and 

fl.n(x) =Fu„(x) -F l:n (x - 1) 

= (q n ) X (l-q n ), * = 0,1,.... (3.6.1) 

Thus, X 1:n is a GeometricG - q") random variable. In other words, X and 
X 1:n come from the same family of distributions. This is true for the 
exponential parent also. 

The distributions of higher order statistics are not so nice for the geomet- 
ric distribution. Now, for the sample range W„, from (3.5.2), we have 

W = 0) = i(pq x )" =p n /(l~q n ), (3.6.2) 

x-0 

and from (3.5.3) we get 
P(W n = w) 

= X :{(q x -q x+w+1 ) n -(q x+1 -q x+w+1 ) n 

* = 0 

~(q x ~ q x+w )" + ( q x+l ~ 9 X+M ')'’) (3-6.3) 

= Y3^r{( 1 - ^ +1 )" - (1 - q w ) n ~ 4(1 - o" - 0 - 

w > 0. (3.6.4) 
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Thus, the pmf can be easily evaluated for a given p and n, from which 
we can obtain E(W n ) and var (W n ). When n = 2, we get P{W 2 = w) = 
2pq w /{\ + q), w > 0. Hence, it follows that E(W 2 ) = 2q/{p(l + q)) 
and E{W 2 (W 2 - 1)} = 4 q 2 /{p 2 {l + q)}. This yields varfW^) = 2qil + q 2 )/ 

{pH 1 + q) 2 }- 

Finally, let us look at the joint distribution of X Vn and W n . Since 
P{Xun=X' K = w)=P{X Un =X, X n:n =x + W), 

we have 

P(X t:n =x,W n = 0) = {/(*)}" 

= [PQ x } n 

= q nx {\-q n ){p n /{\-q n )} 

= P(X Vn =x)P{W n = 0), (3.6.5) 

on recalling (3.6.1) and (3.6.2). For w > 0, 

P(X i:n = x ,W n = W ) = [F(x + w) - F(x -)]" - [F(x + w) - F{x)] H 

-[F(jc + w-) - F( x -)]" 

+ [F{x + w-)-F{ x)]\ 

which is nothing but a single term of the sum appearing in the right-hand 
side of (3.6.3). Quite easily, it can be seen that 

P(X i:n =x,W n = w)=P(X i:n =x)P(W n = w), w> 0, (3.6.6) 

where the two marginal probabilities on the right-hand side are given by 
(3.6.1) and (3.6.4), respectively. From (3.6.5) and (3.6.6) we can declare that 
X i:n and W n are independent random variables. In fact, the geometric 
distribution is the only discrete distribution (subject to change of location and 
scale) for which X x . n and the event {W n - 0} are independent. For a sample 
of size 2, we prove this assertion in Exercise 20. 


3.7. ORDER STATISTICS FROM A WITHOUT-REPLACEMENT SAMPLE 

The discussion of discrete order statistics has so far assumed that the X’s are 
independent and identically distributed. While sampling from a finite popula- 
tion, this assumption is valid if the sample is drawn at random with replace- 
ment. If it is drawn without replacement, while the X’s are still identically 
distributed, they are no longer independent. Let us now see what happens in 
that situation, assuming that the population values are all distinct. 
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Let Jt, < x° < ••• < Xn be the ordered distinct population values mak- 
ing up the population of size N. Then on using a multivariate hypergeometric 
type argument, we can write, with n ^ N, 



izk^N-n+i, (3.7.1) 


and the cdf of X i n is given by 

F i n ( X ° k ) = P ( at least i of the n sampled values do not exceed jc“) 



where for integers a > 0 and b, (£) is interpreted as 0 whenever b < 0 or 
b > a. 

The joint pmf of X i n and X j n is given by 



i <,k <1 <,N - n + j and 1-k^.j -i. 

(3.7.2) 


Closed-form expressions for the first two moments of order statistics can 
be obtained when the population values are labeled 1 through N ; that is, 
when x k = k. Then we will be sampling (without replacement) from a 
discrete uniform distribution. We now obtain these moments for this special 
case by introducing exchangeable random variables Y v ...,Y n+l defined by 

y r = * r:n -* r _ 1:n -l, (3.7.3) 


where X 0 .„ = 0 and X n+l .„ = (N + 1). 

Then, for nonnegative integers y„ y 2 , . . . , y„+i such that L"l} y r = N - n. 


P(Y^y u Y 2 = y 2 ,...,y n+1 = y n+1 ) 

— P{ X^ :n = y, + l,X 2:n = yi +y 2 + 2,..., X n . n — N — y n+l ) 

1 


(?) 


(3.7.4) 


From (3.7.4), it is evident that Y u ...,Y n+l are exchangeable random vari- 
ables. We now use arguments involving symmetry to obtain the means, 
variances, and covariances of the l^-’s. First note that T."l{Y r =* N - n, and 
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hence 

E(Y l ) = (N-n)/(n + 1). (3.7.5) 

Further, since vaKE"^/^) = 0, we have 

{n + 1 )a 2 + n{n + 1 )a l2 = 0, (3.7.6) 

where <r\ = vaKy,) and <r 12 = cov(y,, y 2 ). Now, (3.7.3) can be rearranged in 
the form 


r=l 


Thus, on summing over i, (3.7.7) yields 


Hence, 


" " n(n + 1) " 

E*, s E*/.„ = + Ery„_ r+1 . 

i=l i=I Z r=I 


a, 2 E r 2 + 2 o- 12 £ E « = var ( E 

r= 1 r= 1 j = r + I \ i = 1 


(3.7.7) 


(3.7.8) 


Since we are sampling without replacement, vaKE^^,) = n(N - n)a 2 / 
( N — 1), where a 2 = (N 2 - 1)/12 is the variance of the discrete uniform 
distribution. Further, the sums on the left-hand side of (3.7.8) have compact 
forms. Using these, (3.7.8) can be expressed as 


n{n + l)(2n + 1) n(n - l)(n + l)(3n + 2) 


-°i + 


12 


M2 


n(N+ 1 )(N-n) 
12 


(3.7.9) 


From (3.7.6) and (3.7.9) we obtain 


, n(N+l)(N-n) 

0-1 = '( 4. and 0-12 = (3-7.10) 

( n + !)(« + 2 ) 


Now we are in a position to compare fi i:n ’s and cr, y .„’s using (3.7.7). First, 
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from (3.7.5) we conclude 


H i:n = i(N+ l)/(n + 1). (3.7.11) 

Next, since var(A' i;n ) = iaf + i(i - lV, 2 , we obtain from (3.7.10) 


2 i(n - i + l)(N + l)(N - n) 
' (n + l) 1 2 (n + 2) 


(3.7.12) 


Now for i < j, 


<r U :n = + i(i - 1)^12 = 


i(n-j + 1 )(N+ 1 )(N-n) 

( n + l) 2 (n + 2) 


(3.7.13) 


From (3.7.12) and (3.7.13), one can compute p it]:n , the correlation be- 
tween X i:n and X j n . Clearly, 


Pi,jn 


i(n~j+ 1) 
j(n ~ i + 1) ’ 


i Zj. 


(3.7.14) 


This matches the correlation coefficient of X i n and X J:n from the continu- 
ous uniform parent. The similarity goes beyond that. These discrete order 
statistics also behave like a Markov chain. This is in contrast with the 
situation where sampling is done with replacement. Note that in that case we 
could not find closed-form expressions for the moments of order statistics 
either. 

In our discussion we have introduced exchangeable random variables in 
order to obtain the means and covariances of the order statistics. But, on 
employing some tricky algebra, we can also obtain them directly by using 
their pmfs and joint pmfs given by (3.7.1) and (3.7.2), respectively. That is 
pursued in Exercises 21 and 22. 


EXERCISES 

For Exercises 1-4 assume that a fair die is rolled three times. Let X t denote 
the outcome of the ith roll for i — 1 to 3. 

1. (a) What is the probability that the outcomes of the three rolls are the 
same? 

(b) Find P(X { < X 2 < X 3 ). 
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2. Let Z, = minCA',, X 2 ) and Z 2 = max(A' 1 , X 2 ). 

(a) Find the joint pmf of Z, and Z 2 by looking at the sample space of 
the experiment representing the first two rolls. 

(b) Find the marginal pmfs of Z, and Z 2 . 

(c) Find the correlation between Z, and Z 2 . 

3. Let y, denote the smallest value, Y 2 denote the median value, and y 3 
denote the largest value among the outcomes of the three rolls. 

(a) Show that the pmf of y, can be expressed as 





y, = 1,2,.. .,6. 


(b) Evaluate g(y,) explicitly and use it to obtain the mean and variance 
of Y v 

(c) What is the pmf of y 3 ? 

4. As defined in Exercise 3, let Y 2 denote the sample median. 

(a) Show that the pmf of Y 2 can be expressed as 


2 (y 2 -i)(6-y 2 ) 

g(y 2 ) = — + 


27 


36 


y 2 = 1,2,. ..,6. 


(b) Use the symmetric pmf in (a) to obtain the first two moments of Y 2 . 

5. Let X be a Discrete Uniform [1,5] random variable. 

(a) Compute the pmf of X i n for all i, 1 <i<n, for n = 2 and for 
n = 5. 

(b) Find the means and variances of X i 5 for all i, i = 1 to 5. 

6. By using a multinomial argument, show that when X is discrete, and 
i <j, the joint pmf of X i;n and X j n , f ltj , n {x h Xj), is given by 


f ,=0r 2 =0 r,=0 r 4 =o O' - 1 - r { ) !(1 + + r 3 ) !(j - i - 1 - r 3 - r 4 ) ! 

(1 + r 2 + r 4 )\(n - j - r 2 )\ 

X{F(x r )}‘- l - r mx i )} l+r ' +r iF(x r ) - F(x i )} y ' , ~ 1 ‘ r ’“ r< 
X{/(x y )} 1+f2+f4 {l - F{x j )} n ~ i - r \ 



58 


DISCRETE ORDER STATISTICS 


when Jtj < Xj. Also show that when x t — x jt it takes the form 

. - ■ ■ -> n !(F(x,-))'-'- f '(A^)r' "■(! - fuor’-'’ 

(' ~ 1 - - i + 1 + r, + r 2 )!(n - y - r 2 ) ! 


7. Let X be a Discrete Uniform [1, N ] random variable. 

(a) Determine fi l:2 , a i -.2 and Pi, 2 : 2 - 

(b) Determine variances of X x . 3 and X 3 . 3 . 

8. Prove Lemma 3.4.1. 

[Hint: There are at least two approaches. The first one uses the mono- 
tonicity of (1 - u)*~ r and the mean value theorem of integral calculus. 
The second approach is a statistical one. Note that (3.4.2) is equivalent to 
the statement that cc/vih^V), h 2 (V)) is negative where h x is increasing 
and h 2 is decreasing and V is a (continuous) uniform (a, b) random 
variable.] 

9. Suppose X is either degenerate or has a two-point distribution. Show 
that X in ’s form a Markov chain for each of these parent distributions. 

10 . Suppose S has at least three points in its support. Show that 

(a) (3.4.2) holds if jc,_i = Jt, = Jt i+1 . 

(b) the inequality in (3.4.2) is reversed if either <*, = *,+! or 

11 . Even though the order statistics from a discrete distribution do not form 
a Markov chain, conditioned on certain events, they do exhibit the 
Markov property. 

(a) Show that, conditioned on the event that the observations are dis- 
tinct, the sequence of order statistics from a discrete parent form a 
Markov sequence. 

(b) Can you think of other events E, such that conditioned on E, the 
X i n ’s form a Markov chain? 

(Nagaraja, 1986a) 

12 . Let X have discrete uniform distribution over the first N natural 
numbers. 

(a) Determine E(W 3 ). 

(b) When N = 5, compute the pmf of W 3 and W 5 . 
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13. Let X be an integer-valued random variable having the support [a, b]. 
Then, show that, for w > 0, 


b-w n — 1 n-r n | 

W - *> - £ £ £ TH Hn ' -r- i)! 


x[/( JC )] r [/( JC + M')] 1 [f(x + w - 1) -F(jc)]' 


Verify that this expression matches (3.5.3). 


(Burr, 1955) 


14. When X has a discrete uniform distribution over [a, b], where a and b 
are integers, obtain an expression for the pmf of the spacing W i i+l:n . 
[Hint: Find / > (W' I + , = w) for w > 0. P{W i i+ { = 0) is obtained by 
complementation.] 

15. Let A” be a Geometric(p) random variable. Show that for any j, 
2 <>j <n, X l:n and 1 V\j :n are independent. Are X 2 n and W 2 n:n 
independent? 


16. (a) Compute the pmf of W 2 while sampling from a Bin(4, p) population 

for the following values of the parameter p\ 0.25, 0.50, and 0.75. 

(b) Find E(IV 2 ) and vaKW^) for each of the three values of p used in 
(a). 

17. (a) Verify (3.3.10). 

(b) When the parent distribution is Geometric( p), show that (3.3.10) 
reduces to the following relation: 




(Balakrishnan, 1986) 

18. Let A” be a nonnegative discrete random variable with support S and 
pmf fix). For x e S, the failure or hazard rate function is defined by 
hix) = P(X = x\X >x)= f(x)/{ 1 - Fix - )}. This is analogous to the 
absolutely continuous case discussed in Exercise 2.15. 

(a) Express the failure rate function of X l:n in terms of hix). 

(b) Show that X Un is IFR (DFR) if and only if (iff) X is IFR (DFR). 


19. This exercise shows that among discrete distributions whose support is 
the set of nonnegative integers, the geometric distribution is the only 
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distribution for which the failure rate is a constant. This is a simple 
characterization of the geometric distribution. A detailed discussion of 
the role of order statistics in characterizing the parent cdf F is presented 
in Chapter 6. 

(a) Let A' be a Geometricfp) random variable. Show that its failure 
rate, h(x), is a constant. 

(b) Suppose A is a nonnegative integer-valued random variable. If h(x) 
is a constant, say c (> 0), then show that A' has a geometric 
distribution with parameter p for some p, 0 < p < 1. How are c and 
p related? 

20. Suppose the parent distribution has support S = {0, 1, 2, . . . }. Show that, 
if A 1:2 and the event {W 2 = 0} are independent, the failure rate function 
must be a constant. This, in view of Exercise 19(b), implies the parent 
distribution must be geometric. In Section 3.6 we noted the converse, 
namely, if we sample from a geometric population, X l:2 and the event 
{W 2 = 0} are independent. Hence this independence property character- 
izes the geometric distribution in the class of parent distributions with 
support S. For sample size n > 2, the proof of the characterization is 
harder and may be found in Srivastava (1974). 


21. Suppose the population is made up of N distinct units. We draw a 
random sample of size n(<,N) without replacement. Let A t denote the 
number of population units that do not exceed A i;n , the ith order 
statistic of our sample. 

(a) Determine the pmf of A { . 

(b) Determine E(A i (A i + 1) • • • (/l, + r - D), r s 1. 

(c) Use (b) to obtain the mean and the variance of A t . 

(d) If the population units are in fact the numbers 1, 2, . . . , N, how is A, 
related to A, . „? 

(Wilks, 1962, pp. 244) 


22. Suppose we pick a random sample of size n without replacement from a 
discrete uniform distribution over the integers 1, 2, . . . , N (n ^ AO. Di- 
rectly using the joint pmf of A,.„ and X j:n , determine (r U n when i <j. 
[Hint: Find E( X i:n ) and E{X i n {N - A ;:n )}.] 


23. Let us define B t N = A ( /N, where A,’s are as defined in Exercise 21. 
(a) Show that 


lim 

N -><*> 




T(n + l)T(i + r) 
T(i')r(n + r + 1) ’ 


r £ 1. 


[Hint: Use Exercise 21(b).] 
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(b) Show that B t N converges in distribution to that is, B i N -4 U i:n , 
as N -* oo. (Hint: Use the convergence of the moments.) 

(Wilks, 1962, pp. 245) 

24. Let X 1 ,...,X n be independent and X t be Geometric( p t ), i = 1 to n 
where p/s are possibly distinct. 

(a) Determine the distribution of X l .„. 

(b) Determine whether X l:n and W n are independent. 

(c) Show that there exists a random variable D, possibly degenerate, 
such that X l n ^*D as n -* oo. Determine possible distributions 
for D. 

[Hint: Look at the possible limits for P(X > jc)]. 




CHAPTER 4 


Order Statistics from some Specific 
Distributions 


4.1. INTRODUCTION 

In the last two chapters, we discussed some basic properties of order 
statistics from arbitrary continuous and discrete populations. In this chapter, 
we consider some specific distributions (both discrete and continuous) and 
study order statistics in more detail. In particular, we consider the Bernoulli, 
three-point, binomial, and Poisson distributions in Sections 4.2-4.5, respec- 
tively, and discuss the distribution and joint distribution of order statistics 
and also the derivation of means, variances, and covariances. In Sections 
4.6-4.9 we consider the exponential, uniform, logistic, and normal distribu- 
tions, respectively, and establish some interesting distributional results satis- 
fied by order statistics. In addition to deriving explicit expressions for the 
single and the product moments of order statistics, we further present some 
simple recurrence relations satisfied by these quantities. Finally, in Section 
4.10 we give a discussion on the computer simulation of order statistics. We 
describe how some of the distributional results established earlier in Sections 
4.6 and 4.7 could be used to optimize the simulation process. 

4.2. BERNOULLI DISTRIBUTION 

Let us consider the Bernoulli population with pmf 

?(X = 1) = 7r and P( A" = 0) = 1 - 7 T, 0<7T<1 

In this case, it is easy to observe that 

P(^ i:n = 1) = P(at least n - i + 1 X r ’s are 1) 

- i (")*'o 

r-n-i + l 

= < (say) 


(4.2.1) 

63 
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and 

P(*,, n = 0) = l-<. 

So, we observe that the ith order statistic in a sample of size n from the 
Bernoulli population is also a Bernoulli random variable with probability of 
success tt* as given in (4.2.1), and hence we immediately have 

Mi:n = < and a U:n = Tr*(l-TT*). (4.2.2) 

Similarly, we find the joint probability mass function of X i:n and X j:n 
(1 <i <j <. n ) to be 

P(* ( . ;n = 0, - 0) = ?(X j:n = 0) = 1 - tt/, 

P(^ :n = l,^ ;n = l) = P(^,,, = l)=7rf, 


and 

P(A’,. n = 0, X j:n = 1) = P(at least n - j + 1 and at most n - iX r ’ s are 1) 
= <-<• (4-2.3) 

From (4.2.3) and (4.2.2), we immediately obtain 

= v? and a iJ n = tt*(1 - irf), \Zi<j<,n. (4.2.4) 

From the joint pmf of X i:n and X j:n in (4.2.3), we may also derive the 
probability mass function of the (/, ))th quasirange W t j:n = X j n - X t . „ 
(1 <.i <} <. n) to be 

m,j.n = 1) = P (Xi.n = 0, = 1) = < - < 

and 

P (W U :n = 0) = P(*,, n = X hn = 0) + P(* I;n = X j:n = 1) 

= 1 — TT* + TT* . (4.2.5) 

We thus observe that the (/, j)th quasirange H ' </;n from the Bernoulli 
population is also a Bernoulli random variable with probability of success 
tt* - tt*, and hence 


and 


var(H^ /:n ) = (tt/ - 0(1 - irf + vf ), 1 £ i <j £ n. (4.2.6) 
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In particular, we observe that the sample range W n = X n:n - X l:n from a 
Bernoulli population is again a Bernoulli random variable with probability of 
success v* - v* = 1 - tt" - (1 - t tT, and hence 

,E(^) = 1 - 7T" - (1 - 77-)” 


and 


var(B') = {tt” + (1 - - tt" - (1 - tt)"}, n > 2. (4.2.7) 


4.3. THREE-POINT DISTRIBUTION 

In this section we shall consider the three-point distribution with support 0 
(failure), 1 (waffle), and 2 (success), and with pmf 

P(A'=0)=tt 0 , P(A'=1)=tt 1 and P(A'=2 )=tt 2 , 

with v 0 + n l + w 2 = 1. (4.3.1) 

Proceeding as we did in the last section, we can derive the probability 
distribution of X i:n (1 < i < n). For example, we observe that 

P(A' i;n = 0) = P(at least iX r ’s are 0) 

- E(")»5(i 

r = i 

= <«-,+ 1 ( say), (4.3.2) 

where tt* , is the expression of irf in (4.2.1) with tt replaced by v 0 . We 
similarly observe that 

P(X i:n = 2) = P(at least n - i + 1 X/s are 2) 

- i: 

r=n -i + 1 

= 7r *i (say), (4.3.3) 

where tt*, is the expression of n* in (4.2.1) with tt replaced by tt 2 . From 
(4.3.2) and (4.3.3), we immediately have 

P(* I:fl =l) = l , -tt!,,.. 


(4.3.4) 
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Note that, by direct probabilistic arguments, we can also write 


P(X i:n = 1) = P(at least one X r is 1, at most i - 1 X r ’s are 0, 
and at most n - i X r 's are 2) 

= E E Kexactly rX’s are 1, exactly s^’s are 0, 
r s and exactly n - r - s X’s are 2) 


n: 


n min(i-l,n-r) 


-r-j 

IT j7r 0 7r 2 


(4.3.5) 


Thus, we observe that the rth order statistic in a sample of size n from the 
three-point distribution in (4.3.1) also has a three-point distribution with 
probability of failure ttq „_ j+1 , probability of waffle 1 - 7ro n _ i+ , - it*,!* a °d 
probability of success ir£ ; . From Eqs. (4.3.2)-(4.3.4), we also immediately 
obtain 


- <i) + 2<, = 1 - + <, (4.3.6) 


and 


= (1 " <„-, + ! - *2,i) + 4 ^2 ,i “ (1 " <«-, + ! + </) 


= ’’'o.n-. + l + 7T*., - (< n -i + l “ *2 .,) 


(4.3.7) 


Proceeding similarly, one may derive the joint probability mass function of 
X i:n and Xj :n (1 < i < j < n) and thence an expression for /i, /:n and 


4.4. BINOMIAL DISTRIBUTION 

In this section, let us consider the bionomial population with pmf 

P(* = jc) = (*)p*(l ~P) N ~ X , * = 0,1 N, (4.4.1) 


and cdf 

F( jc) = P(A r £x) = E (?)p r (l ~P) N ~ r ’ * = 0,1 N. (4.4.2) 

r - 0 ' ’ 

Then, from Eqs. (3.2.1) and (3.2.2) we can write at once the pmf of X t . n 
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(1 < i ^ n) as 
/,,„(*) = P(* i:- =x) 

r = i 

-{F( X - l)} r {l -F( X - l)}" - "], x = 0,l (4.4.3) 
with F(- 1) = 0. Alternatively, from Eq. (3.2.3) we can write 

/,:„(*) - i + 1) -I Fix - 1 + !)- x = 0,1,..., TV, 

(4.4.4) 

where /„(a, h) is the incomplete beta function defined by 

7 -(«. 6) = ■^7~T7 dt. (4.4.5) 

B(a,b) J o 

We can write the cdf of X i n (1 < i < n) as 
F l: „(x)-¥{X i:H zx) 

= E(r){ f (^)} r { 1 - f (^))' ,_r . * = 0,1,..., AT. (4.4.6) 

r = i‘ 


From (4.4.6) we obtain in particular that 

*■.:■(*) = L ( n r ){F( X )Y{\ - F( X )} n - r = 1 — {1 - F(x)}", 

r-1 

* = 0,1,..., AT, (4.4.7) 
and 

= { f (*)} n . * = 0,1 (4.4.8) 

Next, from Eqs. (3.2.6) and (3.2.7) we can write the first two moments of 
Xi-.n as 



1 < i < n. (4.4.10) 
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From Eqs. (4.4.9) and (4.4.10), with the use of Eqs. (4.4.7) and (4.4.8), we 
obtain in particular that 


Ml = E {1 

x = 0 

M n;n = E [l - {^(*)}"]. 

Jt-o 

= 2 E *{1 -^(*))" + Ml: 

jr = 0 


and 

M®, = 2 E'*[i - W*)}"] +M, 

jr-0 


By making use of the above expressions, Gupta and Panchapakesan (1974) 
computed the mean and variance of X l:n and X n:n for various choices of n, 
N, and p. For example, in Table 4.4.1 we have presented some of these 
values. 

We may similarly find the joint cdf of X i n and Xj.„ (\ <. i < j <. n) as 


F i j :n (x i ,x j )-F j:n (x i ) = E("){^( *;)}'{! - J 7 ^/)}""', 

r-i 

whenXj^Xy, (4.4.11) 

and when x, < x jt 


j n -j 

= E E 


n\ 


Zist ' 0 r!(n-r -*)!*! 


{^(*,))'{^(*/) - *{i - f( X j)Y 


n n~r 


+ (1 - s in ) E E 


.-o^(»»-r-i)!i! 
x{F(x ; )-F(x ) )}' , " r ■ , {l-F(x / )}^ 




(4.4.12) 


where S in is the Kronecker delta. We may then find the joint pmf of X i:n 
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Table 4.4.1. Means and Variances of Extreme Order Statistics 
for the Binomial BinUV,p) Population 

N 

P 


n 

= 5 



n = 

10 


Pin 

Pn.n 

a l, 1 : n 

n : n 

Ml 

M/t : n 

cr l,l:n 

Vn.n-.n 

1 

0.1 

0.0000 

0.4095 

0.0000 

0.2418 

0.0000 

0.6513 

0.0000 

0.2271 


0.2 

0.0003 

0.6723 

0.0003 

0.2203 

0.0000 

0.8926 

0.0000 

0.0958 


0.3 

0.0024 

0.8319 

0.0024 

0.1398 

0.0000 

0.9718 

0.0000 

0.0274 


0.4 

0.0102 

0.9222 

0.0101 

0.0717 

0.0001 

0.9940 

0.0001 

0.0060 


0.5 

0.0312 

0.9687 

0.0303 

0.0303 

0.0010 

0.9990 

0.0010 

0.0010 

5 

0.1 

0.0115 

1.3188 

0.0114 

0.4544 

0.0001 

1.6544 

0.0001 

0.4202 


0.2 

0.1386 

2.0711 

0.1219 

0.5895 

0.0189 

2.4703 

0.0185 

0.4873 


0.3 

0.4220 

2.7049 

0.2911 

0.6286 

0.1594 

3.1222 

0.1351 

0.4993 


0.4 

0.7985 

3.2634 

0.4301 

0.4301 

0.4615 

3.6736 

0.2814 

0.4595 


0.5 

1.2388 

3.7612 

0.5388 

0.5388 

0.8543 

4.1457 

0.3791 

0.3791 

10 

0.1 

0.1185 

2.1550 

0.1070 

0.7460 

0.0137 

2.6089 

0.0136 

0.6478 


0.2 

0.6649 

3.5140 

0.4263 

1.0281 

0.3301 

4.0586 

0.2391 

0.8521 


0.3 

1.4070 

4.7089 

0.7090 

1.1628 

0.9574 

5.2929 

0.4700 

0.9279 


0.4 

2.2560 

5.8014 

0.9397 

1.1812 

1.7317 

6.3919 

0.6672 

0.9104 


0.5 

3.1886 

6.8114 

1.1019 

1.1019 

2.6183 

7.3817 

0.8186 

0.8186 

15 

0.1 

0.3347 

2.9116 

0.2608 

1.0085 

0.1001 

3.4480 

0.0908 

0.8699 


0.2 

1.3214 

4.8506 

0.7264 

1.4494 

0.8658 

5.5052 

0.4626 

1.1925 


0.3 

2.5200 

6.5926 

1.1332 

1.6815 

1.9398 

7.3026 

0.7873 

1.3396 


0.4 

3.8449 

8.2110 

1.4526 

1.7449 

3.1838 

8.9364 

1.0570 

1.3514 


0.5 

5.2702 

9.7298 

1.6625 

1.6625 

4.5617 

10.4383 

1.2512 

1.2512 

Table adapted from Gupta and Panchapakesan (1974, Ann. 

Inst. Statist. Math. 

Suppl. 8, 

95- 

■113). Produced with permission from the authors and the Editor, Ann. 

Inst. Statist. Math. 


and Xj. „ (1 < / < j < n ) as 



'0, 

x i > x j. 

fi t j: «( 9 ^/) 

F i.,.n(Xi> X i) ~ F i,j:n( X i ~ !>*,)> 

X i = Xj, 

F i,j:n( X i > X j) ~ F iJ:n( X i- My) 



~ F i,j:n( X i> X J - !) + F iJ:n( X i ~ X j ~ K 

), X,<Xj. 

(4.4.13) 

We can then find the product moment M/,y n as 

N N-l N 


f^i,j : n 

^ fi t j : n( ^ ^ 

> x j), 

X 

i * 0 Xj “ 0 Xj = x x i + 1 
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from which we may show that 

N— 1 JC / _1 

M, =Nn Un -( 1 - S Nl ) E E {F(x,) - F(x,)}". 

Xj - 1 jc ( - 0 

From Eqs. (3.5.2) and (3.5.3), we may similarly proceed to derive the pmf 
of the sample range W n = X n:n -X 1:n , obtained earlier by Siotani (1957) 
and Siotani and Ozawa (1958). 

4.5. POISSON DISTRIBUTION 

Let us consider the Poisson population with pmf 

P(Jf = x) = e~ A \ x /x\, x = 0,1,2 (4.5.1) 

and cdf 

F(x) = P(X <x) = e~ K E X/r\, x = 0,l,2 (4.5.2) 

r = 0 

From Eqs. (3.2.1)— (3.2.3), we can then write the pmf of X i:n (1 ^ i ^ n) 
as 

/,:„(*)=?(*:«=*) 

r=*i 

= ^F(i)(*> n - i + 1) ~ i)(*> w — * ■*" 1)> x = 0 S 1, 2 

(4.5.3) 

where F(- 1) = 0 and /„(a, b) is the incomplete beta function defined in Eq. 
(4.4.5). Similarly, we have the cdf of X i:n (1 <. i <. n) as 

*-0,1,2 (4.5.4) 

r—i 

Further, from Eqs. (3.2.6) and (3.2.7) we have the first two moments of X i:n 
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as 


= E {1 — /=■/:«(■*)}» l <i <n, 

jt = 0 


(4.5.5) 


and 


^ = 21^ -Fi.ni*)} +/*<:,. (4.5.6) 

jt = 0 


which, in particular, yields 


and 


M,:„= E{l-f(x))", 

x = 0 

I*.:. - E [l - (f(-t)l"]. 

je = 0 

V-Un = 2 E ^{1 - f(*)}" + Ml:„. 

je = 0 

M ( „ 2 :„ = 2 E t[i - {F(*)}"] + n n 

x~0 


By making use of the above formulas, Melnick (1964, 1980) has computed the 
mean and variance of the extreme order statistics for sample size n = 
1(1)20(5)50(50)400 and A = 1(1)25. Melnick (1980) also discussed how to 
control the error in the computation of these quantities when the infinite 
series is truncated at some point. The values of n l:n and fi n . n are presented 
in Table 4.5.1, for example, for A = 5(1)10 and n = 10,20. 

Furthermore, expressions for the joint distribution of order statistics and 
the product moments can be written similarly to those presented in the 
last section for the binomial distribution. 


Table 4.5.1. Means of Sample Extremes for the Poisson (A) Population 


n 


A = 5 

A = 6 

A = 7 

A = 8 

A = 9 

A = 10 

10 

Ml :« 

1.90970 

2.57418 

3.26691 

3.98173 

4.71442 

5.46200 


/v.« 

8.68131 

10.01314 

11.31785 

12.60116 

13.87604 

15.11834 

20 

/*I:n 

1.38350 

1.97531 

2.59836 

3.25074 

3.93121 

4.61790 


Hn.n 

9.57531 

10.97838 

12.34849 

13.69259 

15.01548 

16.32067 


Table adapted from Melnick (1980, /. Statist. Comput. Simul. 12, 51-60). Produced with 
permission from Gordon and Breach Science Publishers. 






72 


ORDER STATISTICS FROM SOME SPECIFIC DISTRIBUTIONS 


4.6. EXPONENTIAL DISTRIBUTION 


In this section, we shall discuss some properties of order statistics from the 
standard exponential population with pdf 


f(x)=e x , (4.6.1) 


In this case, from Eq. (2.2.3) we have the joint density function of 

^l:n’^2:n X„. n t0 


fl ,2 2 x„)=n\e 0 £ x, <x 2 < ••• <X„< 00 . 

(4.6.2) 


Now let us consider the transformation 

Z \ = nX\ : n’ Z 2 = ( n ~ l)(^2:n — ^l:n) Z n ~ X n :n 

or the equivalent transformation 


Z. Z, 

x Um = -±,x 2:i ,- — + 

n n 


Z 2 Z x Z 2 

— — x n . n = — + — — + 

n - 1 nn n n - 1 


+ Z n- 

(4.6.3) 


After noting that the Jacobian of this transformation is 1/n! and that 
E*,= i(n-i + 1)(*, -*,_,) = E*,, 

i=i i=i i- i 

we immediately obtain from (4.6.2) the joint density function of Z,, Z 2 Z„ 

to be 


fz Z„( Z 1 Z n ) e 




0 <>z 


i> • 


, Z„ < 00. 


(4.6.4) 


Upon using the factorization theorem, it is clear from (4.6.4) that the 

variables Z,, Z 2 Z„ are independent and identically distributed standard 

exponential random variables. This result, due to Sukhatme (1937), is pre- 
sented in the following theorem. 


Theorem 4.6.1. Let X x . n £ X 2:n £ ••• ^X n . n be the order statistics 
from the standard exponential population with pdf as in (4.6.1). Then, the 
random variables Z,, Z 2 Z„, where 


Z,-(n-i + 


* = 1,2 n. 
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with X 0:n s 0, are all statistically independent and also have standard 
exponential distributions. 

Further, from (4.6.3) we have 

X i;n = £z r /(n -r+ 1), i = 1,2, . . . , n, (4.6.5) 

r = 1 


which simply expresses the ith order statistic in a sample of size n from the 
standard exponential distribution as a linear combination of i independent 
standard exponential random variables. It is then clear that the exponential 
order statistics form an additive Markov chain, as noted by Renyi (1953). 

The representation of X i:n in (4.6.5) will enable us to derive the means, 
variances, and covariances of exponential order statistics rather easily (see 
Table 4.6.1.). For example, we obtain 

i i 

to-.n = E(X i:n ) = E E(Z r )/(n - r + 1) = E V(" ~r+l), 

r~ 1 r= 1 

1 < i < n, (4.6.6) 

or i,« :n = var(*,.„) = E var (Z r )/(n ~ r + \) 2 = E 1 /(« - r + l) 2 , 

r=l r= 1 

1 <. i <n, (4.6.7) 
and 

"ij-.n = co \(X l;n ,X j;n ) = E var (Z r )/(n - r + l) 2 

r=\ 

i 

= E l/( « - r + l) 2 = or for 1 < i <j < n. (4.6.8) 

r= 1 

We may similarly derive the higher-order moments of X i n , if needed. 

Alternatively, by making use of the fact that fix) = 1 - Fix) ix > 0) for 
the standard exponential distribution, one can derive recurrence relations for 
the computation of single and product moments of all order statistics. These 
results, due to Joshi (1978, 1982), are presented in the following two theo- 
rems. 


Theorem 4.6.2. For the standard exponential distribution, we have for 
m = 1, 2, . . . , 


..( m ) _ ^ ..(m- 1) 


(4.6.9) 
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Table 4.6.1. Means and Variances of Exponential Order Statistics for n up to 10* 


/I 

i 

Mi:n 

Vi.i-.n 

n 

f 

HH 

Vi.i-.n 

1 

i 

1.000000 

1.000000 

i 

7 

2.592857 

1.511797 

2 

i 

0.500000 

0.250000 

8 

1 

0.125000 

0.015625 

2 

2 

1.500000 

1.250000 

8 

2 

0.267857 

0.036033 

3 

1 

0.333333 

0.111111 

8 

3 

0.434524 

0.063811 

3 

2 

0.833333 

0.361111 

8 

4 

0.634524 

0.103811 

3 

3 

1.833333 

1.361111 

8 

5 

0.884524 

0.166311 

4 

1 

0.250000 

0.062500 

8 

6 

1.217857 

0.277422 

4 

2 

0.583333 

0.173611 

8 

7 

1.717857 

0.527422 

4 

3 

1.083333 

0.423611 

8 

8 

2.717857 

1.527422 

4 

4 

2.083333 

1.423611 

9 

1 

0.111111 

0.012346 

5 

1 

0.200000 

0.040000 

9 

2 

0.236111 

0.027971 

5 

2 

0.450000 

0.102500 

9 

3 

0.378968 

0.048379 

5 

3 

0.783333 

0.213611 

9 

4 

0.545635 

0.076157 

5 

4 

1.283333 

0.463611 

9 

5 

0.745635 

0.116157 

5 

5 

2.283333 

1.463611 

9 

6 

0.995635 

0.178657 

6 

1 

0.166667 

0.027778 

9 

7 

1.328968 

0.289768 

6 

2 

0.366667 

0.067778 

9 

8 

1.828968 

0.539768 

6 

3 

0.616667 

0.130278 

9 

9 

2.828968 

1.539768 

6 

4 

0.950000 

0.241389 

10 

1 

0.100000 

0.010000 

6 

5 

1.450000 

0.491389 

10 

2 

0.211111 

0.022346 

6 

6 

2.450000 

1.491389 

10 

3 

0.336111 

0.037971 

7 

1 

0.142857 

0.020408 

10 

4 

0.478968 

0.058379 

7 

2 

0.309524 

0.048186 

10 

5 

0.645635 

0.086157 

7 

3 

0.509524 

0.088186 

10 

6 

0.845635 

0.126157 

7 

4 

0.759524 

0.150686 

10 

7 

1.095635 

0.188657 

7 

5 

1.092857 

0.261797 

10 

8 

1.428968 

0.299768 

7 

6 

1.592857 

0.511797 

10 

9 

1.928968 

0.549768 





10 

10 

2.928968 

1.549768 


"'Covariances can be obtained by using the fact that (/ > «). 


and 


m 


^=M- ) i: n -t + -M"" 1) . 2*i*n. (4.6.10) 

n 


Proof. For 1 ^ i £ n and m = 1,2,..., let us consider 


- (I - l)!(n - ,)! 





EXPONENTIAL DISTRIBUTION 


75 


which, upon using the fact that fix,) = 1 - Fix,) for the standard exponen- 
tial distribution, can be rewritten as 


„ ( m-0 

r*i : n 


- ( i _ i) T( ! „ - o i /„v-w^ ) r 't 1 - Fix,))- 


i+i 


dx: 


(4.6.11) 


Upon integrating the right-hand side of (4.6.11) by parts, treating x™" 1 for 
integration and the rest of the integrand for differentiation, we obtain for 
1 < i < n and m = 1 , 2 ,..., 


u (m-1) = 

: n 


n\ 


(i - l)!(n - i)\m 


X 


in ~ i + 1) /V{F(x i .)}'-'{1 - F(x,)r , /(x i ) dx, 


-O' - 1) r*r{F(*,)t 2 {l - Fixtf"" fixf) dx, 


. (4.6.12) 


The relation in (4.6.9) follows immediately from (4.6.12) upon setting i = 1. 
Further, by splitting the first integral on the right-hand side of (4.6.12) into 
two and combining one of them with the second integral, we get 

n f xr , {F(x,)} ,_1 {l - F(x,)} n ~' fiXj) dx, 

■'o 

-o - 1) /V{f(*;)r 2 {l - F(x 1 .)} n -'/(A: I ) dx, . 
■'o 

The relation in (4.6.10) follows upon simplifying the above equation. 

□ 

Theorem 4.6.3. For the standard exponential distribution, 

1 

M, + :M,:„, 1 £ i < n - 1, (4.6.13) 

n — i 


(m-l) _ 
r'i : n 


n\ 

(i - 1 ) !( n - i)\m 


and 


n - j + 1 


Mi : n > 


Mi./:n Mi./ — 1 : 


+ 


1 <, i <j < n, j - i 2:2. (4.6.14) 
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Proof. To establish these two relations, we shall first of all write, for 
1 <. i <j <, n, 

» i:n =E(X i:n X? :n ) 


n\ 


(« - l)!(y - * - 1)!(« -»! 

0£x i <x j <’B 

X {l - F(xj )}"' 7 ( Xi)f(xj) dxj dXi 

(,- - 1)10-7'- !)!(,-,-)! )/(*,) 

(4.6.15) 


where, upon using the fact that f(xj) = 1 - F(xj), we have 

I(x t ) = />(*;) - *(* l )r“ l {l - n*j))'- i+ '*j. (4.6.16) 

*i 

Integrating the right-hand side of (4.6.16) by parts, treating dxj for integra- 
tion and the rest of the integrand for differentiation, we obtain, when 

i = i + 1, 

/(*,.) = (n - i)f x,{l - F(xj)} n ~‘~ l f(xj) dXj-Xi{ 1 - ^x,)}" - ', 

X i 

(4.6.17) 


and when j - i S; 2 


/(*,.) = (n -j + 1) fxj{F(Xj) - F(*,)}'-'-‘{l - F(xj)) n - j f(xj)dxj 

-U - i - 1) fxj[F(Xj) - F(x,)} y -'- 2 {l - F(Xj)) n - J+l f(Xj)dXj. 

(4.6.18) 

The relations in (4.6.13) and (4.6.14) follow readily when we substitute the 
expressions of /(*,) in Eqs. (4.6.17) and (4.6.18), respectively, into Eq. 
(4.6.15) and simplifying the resulting equations. □ 
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The recurrence relations presented in Theorems 4.6.2 and 4.6.3 are easy 
to implement through a simple computer program in order to evaluate the 
first m single moments and the product moments of all order statistics at 
least up to moderate sample sizes. One has to be aware, however, of the 
possible accumulation of rounding error that would arise when this recursive 
computation is implemented for large sample sizes. Results similar to those 
in Theorems 4.6.2 and 4.6.3 are available for a number of distributions; 
interested readers may refer to the review article by Balakrishnan, Malik, and 
Ahmed (1988). It should be noted that Theorem 4.6.3 can also be easily 
proved by using Theorem 4.6.1. 

Furthermore, due to the relationship between the geometric and the 
exponential distributions, there also exists a close relationship between the 
dependence structure of order statistics from the geometric distribution and 
those from the exponential distribution. To this end, we may first of all note 
that when Y is Exp(0), i.e., with pdf 

1 

f(y) = «e- y/e , y> o, 0>o, 

u 

then X = [y], the integer part of Y, is distributed as Geometric( p), where 
p = 1 - e~ l/e ; also, the random variables [T] and (Y) = Y - [T] = Y - X, 
the fractional part of Y, are statistically independent. Then, if V,, Y 2 ,...,Y n 
are independent Exp(0) random variables and Y t n denotes the ith order 
statistic from this random sample of size n, we easily observe from the 
representation in (4.6.5) that 

Yi:n = t Y r /(n - r + 1) = £ Y Un . r+u (4.6.19) 

r=l r = 1 


where Y x ;n _ r+1 ’s are independent. A parallel result for the geometric order 
statistics, due to Steutel and Thiemann (1989), is presented in the following 
theorem. 


Theorem 4.6.4. Let X i n denote the ith order statistic from a random 
sample of size n from Geometric( p) distribution. Then, 


r= I 


r+ 1 


L(Y r /(n-r+ 1)) 


(4.6.20) 


where the Y r ' s are independent Exp(0) random variables with 0 = 
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(-log(l - p)) *; and all the random variables on the right-hand side of 
(4.6.20) are independent. 

Proof. First of all, we may note that X l:m = [Y,/nt]. Now 


X,.^[Y), n i[Y i:n ) 


E Y r /(n -r+ 1) 

r-l 


(4.6.21) 


from (4.6.19). Observing then that 


E Y r /(n -r+ 1) 

r-l 


E \Y r /(n - r + 1)] + Z(Y r /(n - r + 1)> 


r-l 


r-l 


= E [Y r /(n -r+l)] + 

r-l 


Z(Y r /(n-r+ 1 )) 

r-l 


we immediately derive from Eq. (4.6.21) that 


Xi.„ ~ E -^l:n-r+l + 


Z(Y r /(n~r+ 1)> 


which is precisely the relation presented in Eq. (4.6.20). The independence of 
all the random variables on the right hand side of (4.6.20) follows readily 
from the independence of the variables [Y] and (Y> [since Y is Exp(0)]. 

□ 


4.7. UNIFORM DISTRIBUTION 

In Chapter 2, we have already seen that the ith order statistic from a random 
sample of size n from the Uniform(0, 1) population has a Beta(i, n - i + 1) 
distribution. Similarly, we have also seen that the ith and ;th order statistics 
jointly have a Bivariate Beta(i, j -/,«-)+ 1) distribution. These distribu- 
tional results were in fact used in Sections 2.2 and 2.3 to derive the means, 
variances, and covariances of uniform order statistics. 

In this section, we shall discuss some additional interesting properties of 
order statistics from the Uniform(0, 1) population. 

Theorem 4.7.1. For the Uniform©, 1) distribution, the random variables 
v \ = u i.n/ u i:n and V 2 = Uj.n’ 1 <. i < ) <. n, are statistically independent, 
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with V x and V 2 having Beta(i,/ - i) and Beta(y, n — j + 1) distributions, 
respectively. 

Proof. From Eq. (2.3.13), we have the joint density function of U i:n and 
U j:n (1 < i < j < n ) to be 


) = 7T 


n\ 


(I - l)!(j - 


0 < U: < U , < 1. 


Now upon making the transformation 


Vl = Ui:n/U}:n 


Vl = Uj:, 


and noting that the Jacobian of this transformation is v 2 , we derive the joint 
density function of V x and V 2 to be 




j-i- 1 


n ! 


0 < u, < 0, 0 < v 2 < 1. (4.7.1) 


From Eq. (4.7.1) it is clear that the random variables V x and V 2 are 
statistically independent, and also that they are distributed as Beta(i, j - i ) 
and Beta(), n — j + 1), respectively. □ 

Proceeding similarly, one can prove the following theorem. 

Theorem 4.7.2. For the Uniform(0, 1) distribution, the random variables 


1 a.’ 


v 2 = 


a*. 

U iy .„ 


= and V,= U irn 

U i, : n 


(1 < i'| < i 2 < • • • < i/ < n ) are all statistically independent, having the 
distributions BetaG' „ i 2 - i',), Beta(i 2 , i 3 - i 2 ), Beta(i,_ „ i, - i,_ x ) and 
Beta(i',, n - i, + 1), respectively. 
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From Theorem 4.7.2, we immediately obtain 

II Vpn j = E j 11 Vf" +m * + ' +m i 

i 

_ Y\E{v mi + mi + +m j) 

= w! t4 f ( i j + m t + m 2+ ••• +mj- 1)! \ 
(n + y= i \ ( ij + m, + m 2 + ••• +m J _ l - 1)! j’ 

(4.7.2) 

where m 0 = 0. From Eq. (4.7.2), we obtain in particular that for 1 < i, < i 2 
< i 3 <i 4 ^n 

E(Up m UfcUp m Up m ) 

n\{m l + ij - l)!(mj + m 2 + i 2 - 1) !(mj + m 2 + m 3 + i 3 - 1)! 

(m x + m 2 + m 3 + m 4 + i 4 - 1)! 

(i'i - l)!(m, + i 2 - l)!(mj + m 2 + i 3 - 1)! 

(m, + m 2 + m 3 + i 4 - 1)!(« + m x + m 2 + m 3 + m 4 )\ 

(4.7.3) 

The first four cumulants and cross-cumulants of uniform order statistics 
obtained from (4.7.3) may be used to develop some approximations for the 
corresponding quantities of order statistics from an arbitrary continuous 
distribution F(x). This method of approximation, due to David and Johnson 
(1954), based on the relation in (2.4.2), is discussed in great detail in 
Section 5.5. 

The following theorem, due to Malmquist (1950), presents an interesting 
distributional property satisfied by the uniform order statistics by making use 
of the property of exponential order statistics given in Theorem 4.6.1. 


Theorem 4.7.3. For the Uniform(0, 1) distribution, the random variables 


V* = 


U,. 


1 : n 


u-,. 


V* = 


2:n 


u 


2 : n 


u, 


3 : n 


V* = 

V n- 1 


t/„ 


• 1 : n 


Un:n 


n ~ 1 


v* = u n , 


and 

(4.7.4) 


are all independent Uniform(0, 1) random variables. 

Proof. Let X i:n <X 2:n < ••• <X n:n denote the order statistics from 
the standard exponential distribution. Then upon making use of the facts that 
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X = - log U has a standard exponential distribution and that -logu is a 
monotonically decreasing function in u, we immediately have 

Xi-.n = ~ logt/ n _ I+ i :n , 1 <i<n. (4.7.5) 

Equation (4.7.5) yields 


V* = 
v i 


U s . 


U i+ U* 


d 



= e ~KX n . i + t 



(4.7.6) 


upon using Theorem 4.6.1, where Y’s are independent standard exponential 
random variables. The independence of the random variables V* 

then readily follows. The fact that these are Uniform(0, 1) random variables 
follows from (4.7.6) when we use the result that e~ Yi 's are uniformly dis- 
tributed. □ 


An alternate simple way of proving Theorem 4.7.3 is to set l = n, i, = 1, 
*2 = 2,..., and i, = n in Theorem 4.7.2 and then to directly use the result 
that (Beta(/, 1))‘ is distributed as Uniform(0, 1). 


4.8. LOGISTIC DISTRIBUTION 


In this section, we shall consider the standard logistic population with pdf 


/( x) = e~ x /(l + e~ x ) 2 , —oo < x <oo (4.8.1) 

and cdf 

F( x) = 1/(1 + e~ x ), —oo < x <oo (4.8.2) 

and discuss some properties of order statistics. 

By noting that F -1 (m) = log[u/(l - u)], we may write the moment-gener- 
ating function of X i n from Eq. (2.2.2) as 




du 


n! 


(/ - l)!(n - *)! 

r(/ + * )r(« - / + i - 1 ) 


r(i)r(n - / + i) 


—B(i + t,n - i - t + 1) 

1 <i<,n. 


(4.8.3) 


where H ) is the complete gamma function. From (4.8.3), we obtain the 
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cumulant-generating function of X i n as 
K i.„(0 = logM i:n (0 

= log T(i + t) + log T(n - i + 1 - t) - log r(i) - log(n - i + 1), 

1 £ i £ n. (4.8.4) 

From (4.8.4), we immediately obtain the mth cumulant of X i:n to be 

- 4r*,:.(0 -4rk>8r(i + <) 

at t -o at i-o 

d m 

+ — • log T(n - i + 1 - t) 
dt v ' i-o 

= ^ (m - 1) (i) + ( - l) m ^ (m_1) (n - i + 1), (4.8.5) 

where ¥ (0) (z) = ^(z) = (d/dz ) log T(z) = r'(z)/r(z) is the digamma 
function, and ^ (1) (z), > I' (2) (z), . . . are the successive derivatives of ¥(z), 
referred to as polygamma functions. From (4.8.5), we obtain in particular that 

M,:, -*£!, = ¥(«■)- *(»-«' + !) (4.8.6) 

and 

<r U:tt = = ^')(«) + - « + 1), (4.8.7) 

where ¥ and ¥ (1) are the digamma and trigamma functions, respectively. 
These functions have been computed rather extensively by Davis (1935) and 
Abramowitz and Stegun (1965); computer programs for the computation of 
these two functions have also been prepared by Bernardo (1976) and 
Schneider (1978). 

Proceeding similarly, Gupta and Shah (1965) have derived an explicit 
expression for the covariance <r,- j . „ in terms of digamma and trigamma 
functions. 

From Eqs. (4.8.1) and (4.8.2), it is easy to note that 

/(*) =F(*){1-F(*)}, -«<*<«. (4.8.8) 

The relation in (4.8.8) has been utilized by Shah (1966, 1970) to establish 
some simple recurrence relations satisfied by the single and the product 
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moments of order statistics. These results are presented in the following two 
theorems. 

Theorem 4.8.1. For the standard logistic population with pdf and cdf as 
in (4.8.1) and (4.8.2), we have for m = 1, 2, ... , 


M ( ri + . = M ( ri - -M ( 1 m ; 1) , 

n 


n > 1, 


(4.8.9) 


.,("■> _ Am) , Am ~ 1) 1 < • < 

f J ’i+l:n + 1 Mi:« ^ ■ Fi.n > ^ ^ ) 


(4.8.10) 


where /4°:n s 1 for 1 < i < n. 


Proof. From Eq. (2.2.2), upon using the relation in (4.8.8), we can write 


= n f xr-'F(x I ){l - F( Xl )) n dx x . 


Integrating this by parts, treating x™ 1 for integration and the rest of the 
integrand for differentiation, we get 


Ml :n 


^-Inf x?F(x x ){ 1 - FX^r'/K) 

m j — oo 

xr{i - 

J — 00 

If l J — 00 

-(« + 1) f *r{i - f(*,)}7(*i) *,1 (4.8.H) 
J — 00 


The relation in (4.8.9) is derived upon simplifying Eq. (4.8.11). 

Next, by using the relation in (4.8.8), let us write from Eq. (2.2.2) 

yr/V-'In^,))'!! - f (*,))-'" dx„ 

1 < i < n. 

Integrating this by parts, treating xf _1 for integration and the rest of the 
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integrand for differentiation, we get 
n\ 

„(m-l) _ 

^ ,:n (i — l)!(n — i')!m 

x((n - I + 1 ) />{F(x,)}'{1 - FlxWfiXi) dx, 

-i/V{F(x I )}'- l {l - F(x l )} H ~ , * l f(x i ) &,.) 

- (i - i).(n -oh. ( ( " + *£*&*,)& - n*dr‘n*> *, 

1{1 /(X,) ^')- (4812) 


The relation in (4.8.10) is derived upon simplifying Eq. (4.8.12). □ 

Theorem 4.8.2. For the standard logistic population with pdf and cdf as 
in (4.8.1) and (4.8.2), we have 


n + 1 


M/.i + l-.n + l 


n — i + 1 


( 


Pi,i + i : n 


n + 1 


u< 2 > 

Pi+l-.n + i 


1 


n - i 


iPi-.n 


and 


n + 1 


Pi.j : n + 1 


n -j + 2 


1 < i <, n - 1 , 
(4.8.13) 


PiJ:n Pi.j-Un 

It ~ j + 2 1 

n + 1 / * u - ,: " +1 " n -j+ l ti ‘ :n 

\ <i <j <,n, j-i^.2. (4.8.14) 


Proof. From the joint density function of X i . n and X j:n in Eq. (2.3.2), we 
may write, for 1 < i <j<n. 


» i:n = E(X i:n X° n ) 


n\ 

( ( -!)!(; -«•-!)!(«-;)! 


/ l *(*i)f(Xi)dx„ 

* — oo 

(4.8.15) 
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where 

7 K) = / {^(*,) - “ F( x j)} n - 1 +l F(Xj) dx , 

X i 

upon using the relation in (4.8.8). By writing 

7 K) =/ { f ( x j) ~ F(Xj)} n ~ 1+l dxj 

x i 

~j i F ( x i) - f, ( Jc i )} J_,_l { 1 ~ F ( x j)} n >+2 

x i 

and then integrating by parts, we obtain for j = /' + 1 

7 (*,) = (« - 0 / *,{! - F(xj)) n ~'~ l f(Xj) dxj 

J X • 

■* i 

-(n-i + l)f *,{l - F(xj)} n ~‘f(xj) dxj 

J X ■ 

■*» 

-■*i^ 7 ( Jc i){l ~ F ( x i)Y ‘ > (4.8.16) 

and for j - i >2 

I(*i) = ( n-j+ 1 )fx J {F(x J ) - F(x i )} j -- , {1 - F(Xj)} n ~ J f(Xj) dxj 

J x, 

- O ' - i - 1 ) fxjiFiXj) - F ( x ( .)}'-'- 2 {1 - F(x J )} n - J + 'f(x l )dx l 

x i 

- (n-j + 2 ) fxj{F(Xj) ~ F(* | .)} /_, " , {1 - F(x,)}"- /+ 7(x,) dx, 

. x i 

-( j ~ i ~ 1 )fxj[F(Xj) - F(*,.)r , '~ 2 {l - F(xj)) n - i+2 nxj)dxj . 

x i 

(4.8.17) 

The recurrence relations in (4.8.13) and (4.8.14) follow upon substituting the 
expressions of /(*,) in (4.8.16) and (4.8.17), respectively, into Eq. (4.8.15) and 
then simplifying the resulting equations. □ 

By starting with the values of fi i:i = E(X) = 0 and /j.', 2 ), = E(X 2 ) = 
tt 2 /3, for example, Theorem 4.8.1 will enable us to compute the means and 
variances of all order statistics for sample size 2, 3, . . . in a simple recursive 
manner. Similarly, by starting with the value of fi x 2 . 2 = E(X x X 2 ) = 0 and 
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Table 4.8.1. Means of Logistic Order Statistics for n up to 10 * 


n 

i 

Mi : n 

n 

i 


i 

i 

0.000000 

7 

i 

2.450000 

2 

2 

1.000000 

8 

5 

0.250000 

3 

2 

0.000000 

8 

6 

0.783333 

3 

3 

1.500000 

8 

7 

1.450000 

4 

3 

0.500000 

8 

8 

2.592857 

4 

4 

1.833333 

9 

5 

0.000000 

5 

3 

0.000000 

9 

6 

0.450000 

5 

4 

0.833333 

9 

7 

0.950000 

5 

5 

2.083333 

9 

8 

1.592857 

6 

4 

0.333333 

9 

9 

2.717857 

6 

5 

1.083333 

10 

6 

0.200000 

6 

6 

2.283333 

10 

7 

0.616667 

7 

4 

0.000000 

10 

8 

1.092857 

7 

5 

0.583333 

10 

9 

1.717857 

7 

6 

1.283333 

10 

10 

2.828968 


* Missing values can be found by the symmetry relation /x, :n - 


t^n—i + 1 : n* 

Table adapted from Gupta and Shah (1965, Ann. Math. Statist. 36, 
907-920). Produced with permission of the Institute of Mathematical 
Statistics. 


using the fact that /*,,« + i ; „ + i = Mi,«-,+ 2 :n + i (due to the symmetry of the 
standard logistic distribution), Theorem 4.8.2 will enable us to compute the 
product moments and hence the covariances of all order statistics for sample 
size n = 2, 3, . . . in a simple recursive way. For example, we have presented 
the values of means, variances and covariances in Tables 4.8.1 and 4.8.2 for 
sample sizes up to 10. Similar tables have been prepared by Balakrishnan and 
Malik (1992) for sample sizes up to 50. 

Results similar to those presented in Theorems 4.8.1 and 4.8.2 can be 
proved for truncated forms of the logistic distribution as well. For these and 
some other developments concerning order statistics from the logistic distri- 
bution, interested readers may refer to the recently prepared volume by 
Balakrishnan (1992). 


4.9. NORMAL DISTRIBUTION 


In this section we shall discuss some important properties of order statistics 
from the normal population. First of all, let us consider the order statistics 
X i:n (1 <. i <, n) from the standard normal distribution with pdf 


1 



-* 2 /2 


<t>(x ) = 


-00 < X < 00 . 


(4.9.1) 
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Table 4.8.2. Variances and Covariances of Logistic Order Statistics for n up to 10* 


n 

i 

i 


n 

i 



n 

i 

i 

a U-n 

n 

i 

i 


1 

1 

1 

3.289868 

6 

2 

5 

0.314798 

8 

2 

6 

0.234796 

9 

4 

6 

0.305223 

2 

1 

1 

2.289868 

6 

3 

3 

0.678757 

8 

2 

7 

0.199582 

9 

5 

5 

0.442646 

2 

1 

2 

1.000000 

6 

3 

4 

0.502298 

8 

3 

3 

0.576257 

10 

1 

1 

1.750100 

3 

1 

1 

2.039868 

7 

1 

1 

1.798479 

8 

3 

4 

0.422171 

10 

1 

2 

0.698437 

3 

1 

2 

0.855066 

7 

1 

2 

0.723663 

8 

3 

5 

0.333326 

10 

1 

3 

0.430922 

3 

1 

3 

0.539868 

7 

1 

3 

0.448112 

8 

3 

6 

0.275479 

10 

1 

4 

0.310962 

3 

2 

2 

1.289868 

7 

1 

4 

0.324031 

8 

4 

4 

0.505146 

10 

1 

5 

0.243115 

4 

1 

1 

1.928757 

7 

1 

5 

0.253667 

8 

4 

5 

0.401428 

10 

1 

6 

0.199531 

4 

1 

2 

0.793465 

7 

1 

6 

0.208385 

9 

1 

1 

1.762446 

10 

1 

7 

0.169184 

4 

1 

3 

0.496403 

7 

1 

7 

0.176813 

9 

1 

2 

0.704838 

10 

1 

8 

0.146843 

4 

1 

4 

0.361111 

7 

2 

2 

0.826257 

9 

1 

3 

0.435270 

10 

1 

9 

0.129711 

4 

2 

2 

1.039868 

7 

2 

3 

0.518475 

9 

1 

4 

0.314261 

10 

1 

10 

0.116157 

4 

2 

3 

0.670264 

7 

2 

4 

0.377749 

9 

1 

5 

0.245775 

10 

2 

2 

0.762446 

5 

1 

1 

1.866257 

7 

2 

5 

0.297171 

9 

1 

6 

0.201761 

10 

2 

3 

0.474405 

5 

1 

2 

0.759642 

7 

2 

6 

0.244966 

9 

1 

7 

0.171104 

10 

2 

4 

0.343956 

5 

1 

3 

0.472872 

7 

3 

3 

0.616257 

9 

1 

8 

0.148528 

10 

2 

5 

0.269720 

5 

1 

4 

0.342976 

7 

3 

4 

0.453287 

9 

1 

9 

0.131213 

10 

2 

6 

0.221830 

5 

1 

5 

0.269035 

7 

3 

5 

0.358864 

9 

2 

2 

0.778071 

10 

2 

7 

0.188382 

5 

2 

2 

0.928757 

7 

4 

4 

0.567646 

9 

2 

3 

0.485139 

10 

2 

8 

0.163699 

5 

2 

3 

0.590527 

8 

1 

1 

1.778071 

9 

2 

4 

0.352157 

10 

2 

9 

0.144736 

5 

2 

4 

0.433653 

8 

1 

2 

0.712975 

9 

2 

5 

0.276365 

10 

3 

3 

0.528071 

5 

3 

3 

0.789868 

8 

1 

3 

0.440811 

9 

2 

6 

0.227421 

10 

3 

4 

0.384963 

6 

1 

1 

1.826257 

8 

1 

4 

0.318472 

9 

2 

7 

0.193208 

10 

3 

5 

0.302949 

6 

1 

2 

0.738319 

8 

1 

5 

0.249174 

9 

2 

8 

0.167946 

10 

3 

6 

0.249782 

6 

1 

3 

0.458165 

8 

1 

6 

0.204612 

9 

3 

3 

0.548479 

10 

3 

7 

0.212513 

6 

1 

4 

0.331705 

8 

1 

7 

0.173560 

9 

3 

4 

0.400684 

10 

3 

8 

0.184934 

6 

1 

5 

0.259882 

8 

1 

8 

0.150686 

9 

3 

5 

0.315762 

10 

4 

4 

0.437368 

6 

1 

6 

0.213611 

8 

2 

2 

0.798479 

9 

3 

6 

0.260607 

10 

4 

5 

0.345659 

6 

2 

2 

0.866257 

8 

2 

3 

0.499214 

9 

3 

7 

0.221890 

10 

4 

6 

0.285865 

6 

2 

3 

0.546413 

8 

2 

4 

0.362941 

9 

4 

4 

0.465146 

10 

4 

7 

0.243769 

6 

2 

4 

0.399331 

8 

2 

5 

0.285120 

9 

4 

5 

0.368453 

10 

5 

5 

0.402646 













10 

5 

6 

0.334261 


* Missing values can be found by the symmetry relation 07 jn = « r n _ j+l n _ j+l . n . 

Table adapted from Balakrishnan and Malak (1992) and produced with permission of the authors. 


In this case, expected values of the sample range X n:n -X l:n were com- 
puted for n z 1000 by Tippett as early as 1925; similarly, Ruben (1954) 
computed the first 10 moments of X„ . „ for n < 50 and Harter (1961, 1970) 
tabulated the means of order statistics for sample sizes up to 400. Tietjen, 
Kahaner, and Beckman (1977) computed the means, variances, and covari- 
ances of all order statistics for sample sizes up to 50. This list (by no means 
complete) gives us an idea about the extensive computations that have been 
carried out concerning order statistics from the standard normal distribution. 
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Derivation of explicit expressions for fi i:n , <r, and cr, ;:n is somewhat 
involved even in the case of small sample sizes. For the purpose of illustra- 
tion, we follow below the approach of Bose and Gupta (1959) to derive exact 
explicit expressions for the means of order statistics in sample sizes up to 5. 
To this end, let us denote 


/„(«) = f {®{<K)) tt -re- xl dx, n- 0,1,2 (4.9.2) 

J -a> ytr 


From (4.9.2), we immediately observe that / 0 (a) = 1. Next, by starting with 

{ 2 \ 2m + 1 j 

<D(ox)--J -j=e~ xl dx = 0, m = 0,1,2,... (4.9.3) 
(since the integrand is an odd function of x ), we get 

2m + 1 /_ 4_i\l 

L (-!)'( jI 7 / 2m + 1 -, ( « ) =°, rn = 0,l,2 


which immediately yields the recurrence relation 

2m + 1 / — i i \ 1 

/ 2 m + .(«)= E (-l) <+, ( 2 ' n . + 1 } 7 / 2|I1 + 1 _ < (fl) > 


m = 0, 1,2 

(4.9.4) 


For example, we obtain from (4.9.4) that 

/.(«) = \ 

and 

/ 3 («) = &(«) ~ K(«) + f/o(«) = f/ 2 («) - l 

For determining / 2 (a), let us consider from (4.9.2) 

= — F <&(ax)xe- (l/2 > x2(2+a2) dx 
da i t ■'-oo 

yfl -oo d ? ? 

j Q(ax) — {e~ (x /2Xfl +2) } 


(4.9.5) 

(4.9.6) 


ir(a 2 + 2) ■'-« 
a 

ir(a 2 + 2)(a 2 + 1) 


i/ 2 


(4.9.7) 
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Upon solving the differential equation in (4.9.7), we obtain 

/ 2 (a) = -tan-'(fl 2 + 1)' /2 , (4.9.8) 

TT 

which, together with Eq. (4.9.6), also gives 

/ 3 («) = ^ tan_1(fl2 + 1)1/2 " ( 4 - 9 - 9 ) 

Now let us consider 

M 2: 2 = 2 / x<&(x)<t>(x) dx, 

* —<x> 

which, upon using the property that (d/dx)<f>(x) = -x<t>(x) and integrating 
by parts, can be written as 

0 ° 00 1 1 1 

H 22 = if <f> 2 (x)dx=[ —e~ xl dx = -^/ 0 (a) = = 0.564190. 

J -oo J -oo tt Vrr vir 

Due to the symmetry of the standard normal distribution, we immediately 
have n I:2 = — /x 2:2 = ~ 0.564190. 

Next let us consider 

M 3 :3 = 3 / x{®(x)) 2 <f>(x)dx, 

J —CO 

which, upon using the property that (d/dx)<t>(x) = -x<f>(x) and integrating 
by parts, can be written as 

oo 3 00 1 3 

^ 3:3 = 6/ <&(x)<l> 2 (x) dx = -=• f <*>(x)-j=e- x2 dx = -^7,(1) 

■'-oo Vir ■'-oo vir vir 

1.5 

= -7= = 0.846284. 
vir 

Due to the symmetry of the standard normal distribution, we then have 
Mi :3 = _ M 3 :3 = -0.846284 and /z 2;3 = 0. 

Let us now consider 

^4.4 = 4/ x{<l>(jc)} 3 <f>(x) di, 

J — 00 
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which can be written as 

(*4,4 - 12/" {*(!)) V(I) * * -r /" - 4-/ 2 (l) 

-'-00 Vir ■'-« Vir Vir 

6 , /- 

= -7=- tan -1 1/2 = 1.029375. 

try ir 

Similarly, we can write 

M 3:4 =12 / x{4>(x)} 2 {1 - <&(x)}«^(x)d* 

— 00 

= 24 r 4>(x)<f> 2 (x)dx-36f {&(x)} 2 <j> 2 (x) dx 

J —ao ^ —oo 

12 18 

= - 7 = J,(l) - -7=/ 2 (l) 
yir yir 

6 18 , _ 

= -=■ j= tan -1 ^2 = 0.297011. 

yir iryir 

Due to the symmetry of the standard normal distribution, we then have 
Mi :4 = -M4:4 = -1029375 and p 2:4 = -/i 3:4 - -0.297011. 

A similar argument will yield 

10 10/3 1\ 

M S!S -^/ J ( l ) -^{^ ,a n- 4 / 2 --} - 1.162964, 

30 40 

M4 :s « -rhW ~ ~rhi\) = 0-495019, 
yir yir 

and then /i, ;5 = — /a 5;5 = —1-162964, /i 2: 5 = _ M 4 :5 = -0.495019, and 
fi 3 . 5 = 0 due to the symmetry of the standard normal distribution. 

Explicit expressions can also be derived for the product moments in small 
sample sizes by proceeding in an analogous manner. However, we refrain 
from that discussion here and refer the interested readers to Godwin (1949), 
David (1981), and Balakrishnan and Cohen (1991). 

The values of p. in and <r, y .„ are presented in Tables 4.9.1 and 4.9.2, 
respectively, for n up to 10. As mentioned earlier, Tietjen, Kahaner, and 
Beckman (1977) provided these values for sample sizes up to 50. 

For the normal distribution, we can establish some interesting properties 
of order statistics by using Basu’s (1955) theorem which states that “a 
complete sufficient statistic for the parameter 8 is independently distributed of 
any statistic whose distribution does not depend upon 8.” 
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Table 4.9.1. Means of Normal Order Statistics for n up to 10* 


n 

1 


n 

i 

Vf.n 

i 

1 

0.000000 

7 

i 

1.352178 

2 

2 

0.564190 

8 

5 

0.152514 

3 

2 

0.000000 

8 

6 

0.472822 

3 

3 

0.846284 

8 

7 

0.852225 

4 

3 

0.297011 

8 

8 

1.423600 

4 

4 

1.029375 

9 

5 

0.000000 

5 

3 

0.000000 

9 

6 

0.274526 

5 

4 

0.495019 

9 

7 

0.571971 

5 

5 

1.162964 

9 

8 

0.932297 

6 

4 

0.201547 

9 

9 

1.485013 

6 

5 

0.641755 

10 

6 

0.122668 

6 

6 

1.267206 

10 

7 

0.375765 

7 

4 

0.000000 

10 

8 

0.656059 

7 

5 

0.352707 

10 

9 

1.001357 

7 

6 

0.757374 

10 

10 

1.538753 


‘Missing values can be found by the symmetry relation /x,. n = 

An — i+ 1 :n" 

Table adapted from Teichroew (1956, Ann. Math. Statist. 27, 410-426). 
Produced with permission of the Institute of Mathematical Statistics. 


Theorem 4.9.1. For the standard normal distribution, we have 

LPi.j-.n = = 1 (4.9.10) 

y= i ;=i 

for i = 1, 2, . . . , n; in other words, the sum of the elements in a row or 
column of the product-moment matrix or the covariance matrix of standard 
normal order statistics is 1 for any sample size n. 

Proof. Let Y 1 ,Y 2 ,...,Y n be a random sample of size n from normal 
M/i,l) population, and Y 1; „ < Y 2:n < ■■■ <Y n:n be the corresponding 
order statistics. Then, upon using the facts that Y = {\/ti)Y." =x Y i is a com- 
plete sufficient statistic for the parameter /i and that the distribution of 
Y i:n -Y does_not depend on /i, we immediately have the result that 
Y i n - Y and Y are statistically independent upon invoking Basu’s theorem. 
As a result, we have coviY i n — Y, Y) = 0, which readily implies 

n 

£ °/,y.« = !’ for i = 1,2,..., n, 

y-i 

since vartY) = 1/n and coviY i n , Y) = (l/yOE^jOj y . n . From this result, we 
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Table 4.9.2. Variances and Covariances of Normal Order Statistics for n up to 10* 


n 

i 

j 

a i,jn 

n 

i 

y 


n 

i 

y 

a ',j:n 

n 

i 

y 

ViJ-n 

1 

1 

1 

1.000000 

6 

2 

5 

0.105905 

8 

2 

6 

0.078722 

9 

4 

6 

0.112667 

2 

1 

1 

0.681690 

6 

3 

3 

0.246213 

8 

2 

7 

0.063247 

9 

5 

5 

0.166101 

2 

1 

2 

0.318310 

6 

3 

4 

0.183273 

8 

3 

3 

0.200769 

10 

1 

1 

0.344344 

3 

1 

1 

0.559467 

7 

1 

1 

0.391918 

8 

3 

4 

0.152358 

10 

1 

2 

0.171263 

3 

1 

2 

0.275664 

7 

1 

2 

0.196199 

8 

3 

5 

0.120964 

10 

1 

3 

0.116259 

3 

1 

3 

0.164868 

7 

1 

3 

0.132116 

8 

3 

6 

0.097817 

10 

1 

4 

0.088249 

3 

2 

2 

0.448671 

7 

1 

4 

0.098487 

8 

4 

4 

0.187186 

10 

1 

5 

0.070741 

4 

1 

1 

0.491715 

7 

1 

5 

0.076560 

8 

4 

5 

0.149175 

10 

1 

6 

0.058399 

4 

1 

2 

0.245593 

7 

1 

6 

0.059919 

9 

1 

1 

0.357353 

10 

1 

7 

0.048921 

4 

1 

3 

0.158008 

7 

1 

7 

0.044802 

9 

1 

2 

0.178143 

10 

1 

8 

0.041084 

4 

1 

4 

0.104684 

7 

2 

2 

0.256733 

9 

1 

3 

0.120745 

10 

1 

9 

0.034041 

4 

2 

2 

0.360455 

7 

2 

3 

0.174483 

9 

1 

4 

0.091307 

10 

1 

10 

0.026699 

4 

2 

3 

0.235944 

7 

2 

4 

0.130730 

9 

1 

5 

0.072742 

10 

2 

2 

0.214524 

5 

1 

1 

0.447534 

7 

2 

5 

0.101955 

9 

1 

6 

0.059483 

10 

2 

3 

0.146623 

5 

1 

2 

0.224331 

7 

2 

6 

0.079981 

9 

1 

7 

0.049076 

10 

2 

4 

0.111702 

5 

1 

3 

0.148148 

7 

3 

3 

0.219722 

9 

1 

8 

0.040094 

10 

2 

5 

0.089743 

5 

1 

4 

0.105772 

7 

3 

4 

0.165560 

9 

1 

9 

0.031055 

10 

2 

6 

0.074200 

5 

1 

5 

0.074215 

7 

3 

5 

0.129605 

9 

2 

2 

0.225697 

10 

2 

7 

0.062228 

5 

2 

2 

0.311519 

7 

4 

4 

0.210447 

9 

2 

3 

0.154116 

10 

2 

8 

0.052307 

5 

2 

3 

0.208435 

8 

1 

1 

0.372897 

9 

2 

4 

0.117006 

10 

2 

9 

0.043371 

5 

2 

4 

0.149943 

8 

1 

2 

0.186307 

9 

2 

5 

0.093448 

10 

3 

3 

0.175003 

5 

3 

3 

0.286834 

8 

1 

3 

0.125966 

9 

2 

6 

0.076546 

10 

3 

4 

0.133802 

6 

1 

1 

0.415927 

8 

1 

4 

0.094723 

9 

2 

7 

0.063235 

10 

3 

5 

0.107745 

6 

1 

2 

0.208503 

8 

1 

5 

0.074765 

9 

2 

8 

0.051715 

10 

3 

6 

0.089225 

6 

1 

3 

0.139435 

8 

1 

6 

0.060208 

9 

3 

3 

0.186383 

10 

3 

7 

0.074918 

6 

1 

4 

0.102429 

8 

1 

7 

0.048299 

9 

3 

4 

0.142078 

10 

3 

8 

0.063033 

6 

1 

5 

0.077364 

8 

1 

8 

0.036835 

9 

3 

5 

0.113768 

10 

4 

4 

0.157939 

6 

1 

6 

0.056341 

8 

2 

2 

0.239401 

9 

3 

6 

0.093363 

10 

4 

5 

0.127509 

6 

2 

2 

0.279578 

8 

2 

3 

0.163196 

9 

3 

7 

0.077235 

10 

4 

6 

0.105786 

6 

2 

3 

0.188986 

8 

2 

4 

0.123263 

9 

4 

4 

0.170559 

10 

4 

7 

0.088946 

6 

2 

4 

0.139664 

8 

2 

5 

0.097565 

9 

4 

5 

0.136991 

10 

5 

5 

0.151054 













10 

5 

6 

0.125599 


’‘‘Missing values can be found by the symmetry relation v i j n = tr„_ J+l 

Table adapted from Sarhan and Greenberg (1956, Ann. Math. Statist. 27, 427-451). Produced with 
permission of the Institute of Mathematical Statistics. 


also obtain that 


n n n 

^ Mi, y: n — ^ Mi:n 52 My:n — 

j - 1 7-1 >”1 

since = «Mi:i = 0. Hence, the theorem. □ 
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Remark 1. Theorem 4.9.1 may alternatively be proved by using the 
relation in Exercise 14 of Chapter 5 with g(ar) = x and noting that 
f'(x)/f(x) = 4>'(x)/4>(x) = -x for the standard normal population. 

Theorem 4.9.2. Let Y 1:n < Y 2 . n < ■ • • <Y n n denote the order statis- 
tics from a normal Mm. o' 2 ) population. Then the statistic 


T = 




(4.9.11) 


with = 0 and S 2 being the sample variance, is independent of both Y 

and S. 

Proqf. The result follows immediately from Basu’s theorem once we note 
that (y, S ) is jointly sufficient and complete for (m, cr ) and that the distribu- 
tion of the statistic T n does not depend on m or a. □ 


Remark 2. Theorem 4.9.2 helps us in the computation of the moments 
of T n (see Exercise 17). It should be mentioned that several tests for possible 
outliers in normal samples are based on statistics of the form T n . The statistic 
(Y n:n - Y)/S discussed in Section 7.9 is one such example. 

As mentioned already in Remark 1, the property that = -x<f>(x) for 
the standard normal distribution can also be exploited to derive some results 
for the normal order statistics. For example, two such results derived are 
presented in the following two theorems due to Govindarajulu (1963a) and 
Joshi and Balakrishnan (1981), respectively. 

Theorem 4.9.3. For the standard normal distribution, we have 


M (2) = 1 + — 

** t ; n / ; 


n\ 


777 E (-O' 


(i- 1)1(11-/)! 


n — i 

j 


i +J 


•Ml ,2 :i+j> 


1 <: i < n. 
(4.9.12) 


Proof. For 1 < i < n, let us consider 
n\ 


" (i-l) /./?(*<■ *<))'"(> - *<*.))""'*(*!) */• 

(4.9.13) 


Upon writing x^ixf) as — 0'(jc,) in Eq. (4.9.13) and then integrating by 
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parts, we immediately obtain 
n\ 


MY 


= 1 + 


(i - l)!(n - i)! 


x 


O’ 


i- 1)E (-!)'(" y *)/ V(*,)<&, 

( n-i ) E (-l) ;+, | rt J 1 )/_ co JC *{®( JC *)} ,+J_I ^ 2 ( JC ') & < 


(4.9.14) 


Now, since f“xj4>(xj)dXj = - j^<j>'(Xj)dxj = <£(*,), we have 
/ -M $ (*.)}V(*,)<fr, = // 'M*i)M*j)dxjdx t 

-oo<x j <Xj<°° 

= Mr+l,r+2:r+2/{('’ + 1 )( r + 2 )} 

= Ml,2:r + 2/{('’+ 0(^ + 2)). 

The last equality follows due to the symmetry of the standard normal 
distribution. The relation in (4.9.12) is derived upon making use of the above 
expression for the two integrals on the right-hand side of (4.9.14) and 
simplifying the resulting equation. □ 


Corollary. By setting i = n in (4.9.12), we obtain for the standard normal 
distribution that 

“ ^Un = 1 + Ml, 2 :„- ( 4 - 9 . 15 ) 

Remark 3. The relations in (4.9.10) and (4.9.15) have been successfully 
used by Davis and Stephens (1977, 1978) to obtain improved approximation 
for the variance-covariance matrix of standard normal order statistics. 


Theorem 4.9.4. For the standard normal distribution, we have 

Em, = 1 + Emi— u:». 1 Zizn, ( 4 . 9 . 16 ) 




and 


E Mi,/:n ~ E l£i£n-l, (4.9.17) 

y-i+i /— i+i 


with s 0 for j £: 1. 


□ 
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4.10. COMPUTER SIMULATION OF ORDER STATISTICS 

In this section, we will discuss some methods of simulating order statistics 
from a distribution F(x). First of all, it should be mentioned that a straight- 
forward way of simulating order statistics is to generate a pseudorandom 
sample from the distribution F(x) and then sort the sample through an 
efficient algorithm like quick-sort. This general method (being time-consum- 
ing and expensive) may be avoided in many instances by making use of some 
of the distributional properties established in preceding sections. 

For example, if we wish to generate the complete sample (x ln , x 2:n ,..., 
x n:n ) or even a Type II censored sample x 2 . x r n ) from the 

standard exponential distribution, we may use Theorem 4.6.1 and avoid 
sorting altogether. This may be done simply by generating a pseudorandom 
sample y i ,y 2 ,...,y r from the standard exponential distribution first, and 
then setting 


x i. n = E y s /( n ~ s + !)- 1 ~ 1 , 2 , . . . , r. (4.10.1) 

S = 1 


If we wish to generate order statistics from the Uniform(0, 1) distribution, 
we may use Theorems 4.7.1 -4.7.3 and avoid sorting once again. For example, 
if we need only the ith order statistic m, . „, it may simply be generated as a 
pseudorandom observation from Beta(i, n - i + 1) distribution. If, in partic- 
ular, the largest observation u n . n is required, it may be generated as v\ /n , 
where v x is a pseudorandom observation from the Uniform(0, 1) distribution. 
This approach, as noted by Schucany (1972), can be used efficiently to 
produce some extreme observations or even a complete sample (u l n , 
u 2:n , . . . , u n:n ). For example, after generating three pseudorandom Uniform 
(0, 1) observations v x , u 2 , and v 3 , we may use Theorem 4.7.3 to produce three 
largest uniform order statistics from a sample of size n by setting 


U = u\/ n u — L}\/ n u\/( n ^ h = iA/ n 2 ) 

“n:n u l > u n-\:n u l u 2 > u n-2:n u \ u 2 u 3 

(4.10.2) 


This method is referred to in the literature as the descending method. 
Lurie and Hartley (1972) provided a similar algorithm which generates the 
uniform order statistics in an ascending order starting from the smallest 
order statistic, and this method is aptly called the ascending method. How- 
ever, it should be mentioned here that the descending method has been 
found to be slightly faster than the ascending method by Lurie and Mason 
(1973) through an empirical comparison. 

Instead, if we require only the smallest and largest uniform order statistics 
from a sample of size n, they may be produced from two pseudorandom 
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Uniform (0, 1) observations v x and v 2 with the use of Theorem 4.7.1 and 
setting 


= "V" 3nd U \:n = ^"(l ' (4.10.3) 

We may note here that this is a combination of the ascending and descending 
methods. 

Suppose some central uniform order statistics, say only (u i:n ,u i+ 

u J:n ), are required; we may simulate these by first generating u J n as a 
pseudorandom observation from Beta(/, n - j + 1) distribution and then 
producing the remaining order statistics through the descending method. 
This simulation procedure due to Ramberg and Tadikamalla (1978), has been 
extended further by Horn and Schlipf (1986). 

Yet another interesting method of generating uniform order statistics is 
due to Lurie and Hartley (1972). This method makes use of the fact that if 

X x ,X 2 X n + x are independent standard exponential random variables, 

then 


r« + lv ’ V' n + 1 V yn+lV 

-A, 


(4.10.4) 


are distributed as U x . n ,U 2:n - U x , n U n:n - U n _ X:n . Now, after generat- 
ing n + 1 pseudorandom Uniform (0,1) observations v x ,v 2 v n + l and 

setting Xj = -log v it we may then produce the uniform order statistics u i:n 
as 


gr-|lQg«r 

E^/logiv’ 


i 


1,2 


,n. 


(4.10.5) 


Note that this method has fewer steps to perform than the descending 
method, but needs one extra Uniform(0, 1) observation. 

The just-described methods of simulating uniform order statistics may also 
be used easily to generate order statistics from any known distribution F(x ) 
for which is relatively easy to compute. Due to the relation in (2.4.2), 

we may simply obtain the order statistics x x . n ,x 2:n x„.„ from the 

required distribution F(-) by setting 

Xi, n = F~\u in ), i = l,2 n. (4.10.6) 

For example, if we seek the order statistics x X:n , x 2: „ x n . n to come from 

the standard logistic population with cdf F(x) = 1/(1 + e~ x ) and inverse cdf 
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F *(u ) = log[u/(l - u)], we will simply set 

*, :n - log | x “ , M n - ), i = 1,2, (4.10.7) 

We refer the interested readers to the books by Kennedy and Gentle 
(1980) and Devroye (1986) for a more detailed treatment of this topic. 


EXERCISES 

1. For the Bernoulli population considered in Section 4.2, find the distri- 
bution of (X i:n +Xj. n ). Find also the distribution of the sample me- 
dian X n . 

2. Derive the probability mass function of the (i,y)th quasirange W t j . n = 
Xj. n - X i:n , 1 < i < j < n, for the three-point distribution discussed in 
Section 4.3. Deduce the distribution of the sample range W n = X n n - 

3. Let Y be Exp(0) with 1/8 = -log(l - p). Show then that X = [Y], the 
integer part of Y, is geometrically distributed with parameter p. Show 
that X and Y - X are statistically independent. Prove further that 
X i n = [ k] . J by showing that their distribution functions are the same. 


4. Consider the right-truncated exponential population with pdf 
f( x ) = -j; e ~ X ’ 0 <x<P u 


= 0 , 


otherwise, 


where 1 - P (0 < P < 1) is the proportion of truncation on the right of 
the standard exponential distribution and P { = -log(l - P ). Then, by 
proceeding along the lines of Theorem 4.6.2, show that for m = 1,2,..., 


m I 1 — P ' 

(a) p\ m) n = -pT.n' ) -[-y- n> 2. 


(b) 


1 


m . _ _ „ 

("■) = _„( m ) -| „(m — 1) _ ,,("■) 

M i — 1 : n — 1 ' "Pi'.n 

n 


1 -p 


Pi/n- 1 - 


2 < i < n — 1 . 


1 m l 1 — P 1 

(c) + -P ( n r n l) - ( -s- I / > , m , n 2. 


(Joshi, 1978) 
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5. For the right-truncated exponential distribution considered in Exercise 4, 
by proceeding along the lines of Theorem 4.6.3 prove that 


(®) Mn-l,n:n Mn — l:n Mn — l:n 

-W | p j{f* lMn-l:n-l ~ M^n — 1 : n — 1 } » « ^ 2. 


(b) M/./ + l:n=M?:n 


+ • 


n — i 


Pi:n p j{Mi,i+l :n - 1 Mi:n-l} 


1 ^ i < n - 2. 


(^) Ai,j:n 

p j{M/,/:n-l Mi.y— 1 :n — l} > 

1 ^ i <; <, n - 1, j - i ^ 2. 

— 1 Mi, n-I:n-l}i 

1 ^ i <, n - 2. 
(Joshi, 1982) 

6. Derive exact explicit expressions for the first two single moments and 
also the product moments of order statistics from the right-truncated 
exponential distribution considered in Exercise 4. 

(Saleh, Scott, and Junkins, 1975) 

7. Consider the doubly truncated exponential distribution with pdf 

/(*) = Y~Q e ~ X ’ Qi^ x ^ p \> 

= o, otherwise, 

where Q and 1 - P (0 < Q < P < 1) are the proportions of truncation 
on the left and right of the standard exponential distribution, respec- 
tively, and Gi = -log(l - Q) and P x = -log(l - P). Then, by denoting 


n-y + ip 

(d) Mi ,n:n M/,n — l:n 4" Mi:n 
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02 = 0 ~ Q)/(P ~ Q ) and P 2 = i\ - P)/(P - Q ), generalize the re- 
sults in Exercise 4 by proving that for m = 1, 2, , 

(a) n\ m \ = Q 2 QT + - P 2 n\ m) n -u n > 2. 

n 

m 

(b) n\ m) n = Q2f.tV.n-l + ~^n l) - PinW-u 2<i<n-l. 

m 

(c) tf) n = ^ 2. 

(Joshi, 1979a) 

8. For the doubly truncated exponential distribution considered in Exercise 
7, show that for 2 < i < n and m = 1,2,..., 


(m) = (m) + 


«-.) 


™£(1 -Q 2 ) n ~ r B(i,r-i + l)/*STr _1> 


+ (1-02)'’"' 


and 

„("0 = u (m) + f n ^ 

Pi:n H-i-Un ^ I / _ 1 I 




-m £ Q‘f'B(r - 1 ,n - i + 2)m < ,"L7: , J-h 

r = 2 


where fi(-, • ) is the beta function. 

(Balakrishnan and Joshi, 1984) 

9. For the doubly truncated exponential distribution considered in Exercise 
7, prove that 

(®) A 1 /! — l,n :n — Mn-l :n + Mn-l :n — :n - 1 — Mn-l :n - l}> 


n > 2. 


(b) M, + — — [m,:„ - nP 2 {fi l i 


n - i 


i + 1 : n — 1 


eft-, )1. 


1 < t < n — 2. 

( C ) A t i,/:n = PiJ-l.n + n _ y + j [^nn - ^lipij.n-l ~ Pi,J - 1 : n - l}] > 

1 ^ i < i <, n - 1, j - i > 2. 

A t i,n:n — Pi,n~ 1 :n + A t i: n ~ n Pl^P\P-i:n~\ ~ Pi.n-l :n-l)> 

1 ^ i ^ n — 2. 
(Balakrishnan and Joshi, 1984) 
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10. Prove Theorem 4.7.2 by following the line of proof given for Theorem 
4.7.1. 


11. For the standard logistic population considered in Section 4.8, derive an 
explicit expression for the product moment fi U:n (1 <. i <j <. n ) as 


PiJ:, 


= + 


j - 1 r— 1 

£ £(-i) 

r—i j== 1 


r+tj r - 1 ]/n)/j - i - r + s\ 

- 1 A r A * I 


XB(s,n - r + l)fij +s - r:n+s - r + (r) E (“ 1 ) , ( #I r ')t77 

x[-^ (1) (/i -/ + 1) + {¥(/i -j + 1) - ¥(/i - i - r + 1)} 
x{¥(/-i-r) -V(n-j+ 1)}]. 

(Gupta and Shah, 1965; Shah, 1966; Balakrishnan, 1992) 


12. Consider the doubly truncated logistic population with pdf 


= 0, 


Qi^x<, P,, 
otherwise, 


where Q and 1-P(O<0<P<1) are, respectively, the proportions 
of truncation on the left and right of the standard logistic distribution, 
and Qi = loglQ/(l - Q)] and P, = log[P/(l - P)]. Then, by denoting 
Q 2 = Q( 1 - G)/(P - Q) and P 2 = P(1 - P)/(P - Q) and proceeding 
along the lines of Theorem 4.8.1, prove that for m = 1,2,... 


(a) = QT + j^q [ p 2i p ? ~ QT ) 

(b) fl!i = pr-j^Q[Q 2 {pr-Qr) 

+ (1-2 C){P,"-M"1}-»*M"‘ ,) 1- 
(C) - QT + JZ-q[p - QT) 

+(2P-i){ A tv?]-Gr}-7PV?; ,) j. 

n ^ 2. 
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(d) = n\ m l + 


P-Q 


Pz{P? - 

2P - 1 , , m 

4 . {„(">) — ) /(" 1 )l _ 

+ 0 1^2:2 Ml:2J ~ ^2: 


-1) 


(e) M ( 2 m U.=M ( ri + . + 


« + 1 


/* - G 
+ 


L_ /..("■) _ „(m) 1 

2P~ 1 , 

— — V2 m) n -M ( ,7>}- 


m 


n(n-l) 


P ( 2 m .n l) 


n > 3. 


(o Mr, ;n+ , = 7 


« + 1 


i(2P - 1) 


m 


n - i + 1 


„(m-D 

run 


nP 7 


(g) Mn”+ ! : n + ! = 


« + 1 


«(2P- 1) 


(if (m) - u (m) 1 

„ _ i + 1 Pi-Un-ll 

"7TT^" + 1 )( p + G "i) 

fl + 1 

/(2P-1)}m^ + 1 + (^ + G-1)mW :( , 

2 ^ i < n — 1 . 


n + 1 

-n(2P - l)}^, + (P + G - 

n >2. 

(Balakrishnan and Kocherlakota, 1986) 


13. For the doubly truncated logistic distribution considered in Exercise 12, 
by proceeding along the lines of Theorem 4.8.2 establish the analogous 
recurrence relations for the product moments /i, yn . 

(Balakrishnan and Kocherlakota, 1986) 

14. Derive exact explicit expressions for the first two single moments and 
also the product moments of order statistics from the doubly truncated 
logistic distribution considered in Exercise 12. 


(Tarter, 1966) 
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15 . Prove Theorem 4.9.1 by using the result given in Exercise 14 of 
Chapter 5. 

16. Prove Theorem 4.9.4. 

(Joshi and Balakrishnan, 1981) 

17. Let X l: „ < X 2n <, ••• <. X„.„ be the order statistics from the stan- 
dard normal distribution, and X k be the average of the k largest order 
statistics given by 


1 A 

X k = T Z X i:n . 

K i-n-k+l 

Then, by using Theorem 4.9.4 show that 

E{X£) = p t (2 / -2n + 2k- l) M ft - ^ • 

K i=n-k + 1 K 

Define the kth selection differential or reach statistic as 

1 " 1 * 

D k = X k — X n — ~r z x i:n --Zx i:n . 

K i-n-k+l n i ~ 1 

Show then that 

E(D k )-E{X k ) and E(D 2 k ) - var(^) + (£(^*)} 2 ~ 


and hence 


var( D k ) = var(**) - -. 

For testing the presence of outliers in normal samples, Murphy (1951) 
proposed the internally studentized version of the selection differential as 


T k = kD k /S, 


where S 2 is the sample variance (refer to Sections 7.9 and 8.6 for a brief 
discussion). By making use of Theorem 4.9.2, show that for m = 1,2 


k m [{n - \)/2\ m/2 T[{n - \)/2\ 

[ r[(n-l+m)/2] 


£(£>?)• 


E(T?) = 
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18. For the Cauchy population with pdf 


/(*) 


1 

ir(l +jc 2 ) ’ 


-00 < X < 00 , 


prove that 


~ Mi-i ;n -i) - 1, 2> <, i < n - 2, 


and more generally 


U ( . m) = (ut™-') - 1) 1 _ (m- 2) m + i < ; < „ _ 

^i:n • r*'i — l:n — 1J Mi:n ’ ® + 1 S I <fl 


m. 


(Barnett, 1966) 


19. For the gamma population with pdf 

1 

/(*) = f^ e x * p \ *^0, p = 1,2,3,..., 


show that exists for m > -p by showing that £(A"") exists for all 
m > -p. Prove then that 


m p 1 

M ( ri = — r(p) E MVr-'VH, n > 2, 

" r = 0 


and 


m 


p-i 


- r( p) E M ( ,-r " p) A !, 2 <i < n. 


r=0 


(Joshi, 1979b) 

20. Let X v X 2 , . . . , X n + l be n + 1 i.i.d. standard exponential random vari- 
ables. Then, prove that the variables 


V. - 


— 1 — y = f — [/ _ " 

’ 2 ’"■’ n v« + 1 v » 


E?= + ,^ 
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are jointly distributed exactly as 

Vu..U 2:h -U u „ U n , n -U n . Un , 

where U i . H > s are the order statistics from the Uniform(0, 1) distribution. 

(Lurie and Hartley, 1972) 

21. Suppose we are interested in simulating the central n - 2i + 2 order 
statistics from the Uniform(0, 1) population. Then, justify that it can be 
done by the following steps: 

1. Generate as a Beta(/i - i + l,i) observation. 

2. Given U n _ i+1:n = u n _ i+1:n , generate U i:n as u„_ i+i:n V, where V is a 
Beta(i, n - 2i + 1) observation. 

3. Given U i:n = u i:n and U n _ i+i:n = u„_ (+1 .„, generate a permutation 

of [U i+l .„ U , as a random sample of size n - 2i from the 

Uniform(u,. n , u n _ i+1:n ) distribution. 

4. Arrange the subsample generated in (3) in increasing order. 

(Horn and Schlipf, 1986) 

22. Instead, suppose we are interested in simulating the i largest and 
smallest order statistics from the Uniform(0, 1) population. Then, justify 
that it can be done by the following steps: 

1. Generate as a Beta(/i - i + 1,0 observation. 

2. Given U„_ t+l:n = u„_ (+1;n , generate U i:n as u n _ i+l:n V, where V is a 
Beta(i, n - 2i + 1) observation. 

3. Given U i:n = u i:n and = u n _ i+Vn , generate permutations of 

[U Un , . . . , U t . v J and {U„_ i+2: U n:n ) as random samples of size 

i - 1 from Uniform (0 ,u i:n ) and Uniform(u n _ l+1:n , 1) distributions, 
respectively. 

4. Arrange the two subsamples generated in (3) in increasing order. 

(Horn and Schlipf, 1986) 

23. With the help of the algorithms in Exercises 21 and 22, explain how you 
will simulate (i) the central n - 2i + 2 order statistics and (ii) the i 
largest and smallest order statistics, respectively, from the Laplace or 
double exponential population with pdf 

f(x) = je~ M , -oo <*<«». 


24. For the half logistic population with pdf 


/(*) = 2e~V(l + e~ x ) 2 , 0 <; * < ®, 
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by proceeding along the lines of Theorem 4.8.1 prove that for m = 

1 , 2 ,... 

(a) /^U= 2 * {/^i 


(n + l)m n — 1 

V ' „(»•- 

Ml :n 


thl = v ~ 7 '"..t" 1-1 ) — - ,,< m ) n > 1 

/*2:n + l Ml : n ~ Ml:n + 1> “ — *• 


1 


(«) M^ 1:n + 1 = 7 


(n + l)m n + 1 

1 — + u (">) 

n-i + l M,:n 2 M, ' 1:n 


n — 2i + 1 


-n< m) 
Mcn + i 


2 < i < n. 
(Balakrishnan, 1985) 


25. For the half logistic distribution considered in Exercise 24, by proceeding 
along the lines of Theorem 4.8.2 prove further that 


(®) Mi',i' + l:n + l Mi':n + 1 ■ . I p\:n .Mi':nl> 

n — i + l V n — i I 


1 ^ / < n - 1. 

(*>) M2, 3:n + l = M&. + 1 + (« + 1 ){m 2 :,i ~ J Mn " ~ l} ’ " * 2 ‘ 

( C ) Mi + l,i+2:n + l = M^ 2 +2:n + l 
n + 1 

^ 1 ) + 1 : « ^ { Mi — 1 , 1 : h — 1 Mi ' :n — 1 } ] > 

2 <, i <. n - l. 


Mi,j':n + 1 Mi,j — l:n + l 

2(n + 1) 


1 


Mi',j:n Mi',/— l:n 


Mi:n 


j:n n-y + r 

1 <i<j<,n, j - i > 2. 


n — j + 2 

( e ) M2,/+l:n + l = M3,j + l:n + l + (« + l)|M;:n ~ ^Ml, j - 1 : n - lj > 

3 <j<n. 

(0 Mi+i .j + lin + l Mi+2, j + 1 : n + 1 

/J + 1 

[2M/:n n {Mi,j'-l:n-l — Mi- l,j-l : n~l)]> 


+ ■ 


/(/ + !)' 


2 < i <j <, n, j - i k 2. 

(Balakrishnan, 1985) 
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26. By starting with the values of = E(X) = log 4, /tf?, = E(X 2 ) = 
ir 2 /3, and Mi, 2:2 "'Mi:i = Oog 4) 2 , employ the results in Exercises 24 
and 25 to compute the means, variances, and covariances of order 
statistics from the half logistic distribution for sample sizes up to 10. 

27. Let X x , X 2 , . . . , X n be i.i.d. Uniform(0, 1) variables, and X„. n = 
max(Xj, X 2 , . . . , XJ and X* „ = X\ /n . Then, show that 

(a) P (X n:n ^x,X* n <y)=x n - i y\ x > y" 

= x n , x <,y n , 0 £x,y <, 1. 

(b) nx n;n zx: :n ) = j^. 

Realize that the largest order statistic in a sample of size n from 
Uniform(0, 1) population can be generated as either x n . n or x*. n defined 
above. 

(Nagaraja, 1979) 

28. Prove the relations in Theorem 4.6.3 directly by using the independence 
of the exponential spacings established in Theorem 4.6.1. 



CHAPTER 5 


Moment Relations, Bounds, and 
Approximations 


5.1. INTRODUCTION 

In the last chapter we derived some recurrence relations satisfied by the 
single and the product moments of order statistics from some specific 
populations like the exponential, normal, and the logistic. These relations 
were derived by making use of the basic differential equation satisfied by the 
population distribution. In this chapter, we first establish some identities and 
recurrence relations satisfied by the moments of order statistics from any 
arbitrary population. We then derive some universal bounds for the moments 
and some simple and useful functions of moments of order statistics, and also 
describe a method of approximating these quantities. 

In Section 5.2 we first give the basic formulas of the single and the product 
moments of order statistics. By using these formulas we derive in Section 5.3 
identities and recurrence relations satisfied by the single and the product 
moments of order statistics. We present a theorem which gives the minimum 
number of single and product moments to be computed in a sample of size n 
for the evaluation of all the means, variances, and covariances of order 
statistics for any arbitrary distribution. We present similar results for the case 
when the population distribution is symmetric. In Section 5.4 we derive some 
universal bounds for the expected value of an order statistic and of the 
difference of two order statistics. We also present improved bounds for these 
quantities when the population distribution is symmetric. Finally, in Section 
5.5 we develop some series approximations for the moments of order statis- 
tics and present the formulas in particular for the means, variances, and 
covariances of order statistics. 
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5.2. BASIC FORMULAS 

Let X U X 2 X n be a random sample of size n from an absolutely 

continuous population with pdf fix ) and cdf Fix), and let X l: „ <, X 2: „ <, 

• • • £X„.„ be the corresponding order statistics. From the pdf of X i:n in 

(2.2.2) , we then have, for 1 £ i £ n and m = 1,2 

M ( /"> = £(*/"„) = f x m f hn ix)dx 

J —00 

- (,• - 1 ) K , ' - ■•)! 0" (f(jr)1 "' (1 - *<*»""'<*> *■ < 5 - 21 > 

As mentioned in Chapter 2, we will denote by for convenience. 
From the first two moments, we can determine the variance of X i:n by 

=^ = var(Z (;n ) =n?) n ~n), n , \<i<,n. (5.2.2) 

Similarly, from the joint density function of Xi.„ and X ] n in (2.3.2), we 
have, for 1 £ i <j <n and m ( , = 1, 2 

~ E{XZj,Xj”t,) 

// xpxpfu .. „i X, , Xj) dx t dXj 

n\ 

= (/- !)!(;-/- !)!(„-;)! 

X // xr‘xp{ Fixrf-'iFiXj) - Fixjf*-' 

x {i - FiXj)}"~ J fi xjfixj) dx ( dxj. (5.2.3) 

Once again, for convenience we will use ix i j:n instead of $’/?„• The 
covariance of X i n and X j n may then be determined by 

“ij-.n = cov(A' ( .„,^.„) = fii J.„ \ < i < j £ n. (5.2.4) 

The formulas in (5.2.1) and (5.2.3) will enable one to derive exact explicit 
expressions for the single and the product moments of order statistics, 
respectively, in many cases. Also, in situations where it is not possible to 
derive such explicit expressions for the moments, the formulas in (5.2.1) and 

(5.2.3) can be used to compute the necessary moments by employing some 
numerical methods of integration. 
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The expressions for the single and the product moments of order statistics 
in Eqs. (5.2.1) and (5.2.3) can be easily modified to the case when the 
population distribution is discrete and written as 


u (m) 

r*i : n 


and 

r*i,j:n 


E(X” n )= E x m f hn (x), l<izn, m = 1,2,..., 

L l <.x^L 2 

(5.2.5) 


E{X™j,Xj"> n ) = 


EE xpxpf^X'.Xj), 

L,< 

1 <,i<j<n, m i ,m j = 1,2,..., (5.2.6) 


where [L l ,L 1 ] is the support of the discrete population distribution; in 
Eqs. (5.2.5) and (5.2.6), /,.„(*) and / u .„(*,, Xj) are the pmf of X i;n and the 
joint pmf of X i:n and X j:n , respectively, as given in Chapter 3. 

Now, by defining the inverse cumulative distribution function of the 
population as 

F~ x (u) = sup{x: F(x) < u}, 0 < u < 1, (5.2.7) 

and by using the relation in (2.4.2), we can unify the expressions of the single 
and the product moments of order statistics and write them as 


i) ), /V-V))V '(i - u)-‘du, 

1 <i <n, m = 1,2,..., (5.2.8) 

and 

a {m ‘' m ‘ ) = E(X m ‘ X m > ) 

r , i y ):n \' x i: n' x j:n ) 

n\ 


( i-l)!(;-i-l)!(«-;)l 

0<u,<u J < 1 

x (1 ~ Uj) n ’ dUi duj, l < i < j < n, m ,, m ) ■ = 1,2,... . 

(5.2.9) 


We shall use the expressions of the single and the product moments in (5.2.8) 
and (5.2.9) in subsequent sections to derive some identities, recurrence 
relations, and bounds for these moments, which then will hold for any 
arbitrary population. 
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S3. SOME IDENTITIES AND RECURRENCE RELATIONS 

In this section we establish some basic identities and recurrence relations 
satisfied by the single moments, the product moments, and the covariances of 
order statistics. These results, in addition to providing some simple checks to 
test the accuracy of the computation of moments of order statistics, can 
reduce the amount of direct computation of these moments of order statis- 
tics, which is highly desirable in cases where these quantities have to be 
determined by numerical procedures. 

From the identities 

£*/?„= E*r> ^i. (5.3.1) 

1-1 1=1 

and 

E E X,”kXp n =EE XrXp, m, m y £ 1, (5.3.2) 

i=i y=i ;=iy=i 

upon taking expectation on both sides, we derive the identities 

E = nE(X m ) = ntfl m> 1, (5.3.3) 

i = i 

and 

E E Hu'-? 1 * = nE (X m ‘ +m >) + n(n - \)E{X m >)E(X m i) 

«-iy=i 

= + n(n - l)M < riVi?i > . m n m y ^ 1- (5-3.4) 

From the identities in (5.3.3) and (5.3.4), we obtain in particular that 

Em,:„ - nE(X) = n#ti:i, (5.3.5) 

i = i 

itf> H -nE(X 2 )-nv® lt (5.3.6) 

i-i 

E E M,.y. n = n Hui + Hn - l)fti:i. 

«=iy=i 


(5.3.7) 
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which, for the case when m, = m 2 = 1, gives 

M 1 , 2 : 2 = : 1 ' (5.3.10) 


By combining (5.3.8) and (5.3.10), we obtain the identity 


n- 1 

E 

f ( n ) 2 

E = 1 2 Ml:- 

(n\ 

1 ~~ 1 2|Ml,2:2- 

(5.3.11) 

i=i j 

=i+i v 7 



Furthermore, from the identities in (5.3.5) and (5.3.7), we get 


n n 

n n 

' n Ur, \ 


E E 

n E E ) 

Xs : n 1 ^ ; :n 1 


1=1 y=i 

< = 1 /= 1 

Vi* 1 \ y = 1 / 



= - Ml;l} 

= n var( X) 






(5.3.12) 


All the just-given identities are very simple in nature and hence can be 
effectively used to check the accuracy of the computation of the single 
moments, the product moments, variances, and covariances of order statistics 
from any arbitrary population. 

In addition, these quantities satisfy some interesting recurrence relations, 
which are presented in the following theorems. 

Theorem 5 .3,1. For 1 < i < n - 1 and m = 1,2,..., 

itiVi.n + (n- i> ( /^ = «/*VTiS-i 


(5.3.13) 
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Proof. From Eq. (5.2.8), let us consider 


ntfZ-i = 


n\ 




( i - l)!(/i — i — 1)! J o 

0- i)!(!i ! - i -l) !( 1<F ~' ( “ >) ’“'~' (1 -“ >, ' l ~' 1 " + “ -»* 


(«■ - 1 ) 1(11 


+ | 1 {F _I (M)) m «'' 1 (l - u) n_, du| 


= «>ft> 1: . + (*-«>£>. 


The last equation follows by using Eq. (5.2.8) and simplifying the resulting 
expression. □ 

The recurrence relation in (5.3.13), termed the triangle rule, shows that it is 
enough to evaluate the mth moment of a single order statistic in a sample of 
size n, if these moments in samples of size less than n are already available. 
The mth moment of the remaining n - 1 order statistics can then be 
determined by repeated use of the recurrence relation in (5.3.13). For this 
purpose we could, for example, start with either or It * s > therefore, 
desirable to express the moment purely in terms of the mth moment of 
the largest order statistics or of the smallest order statistics from samples of 
size up to n. These relations, due to Srikantan (1962), which can then be 
utilized instead of the triangle rule for the recursive computation of the 
single moments of order statistics, are presented in the following two theo- 
rems. 


Theorem 5J.2. For 1 < i ^ n - 1 and m = 1, 2 

- E(-l)'"(r)( -I JU’!- (5.3.14) 

r = i ' ' 

Proof By considering the expression of * n (5.2.8) and expanding 
(1 - w)" - ' in the integrand binomially in powers of u, we get 


. n ! 

(m) = y r _ n 

iV J (« - l)!s!(« - « -s)! ■'o 


j l {F 1 (k)} , "w ,+i 1 du 


n ~' s n! 


u (m) • 
ri+s-.i+s' 
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Upon rewriting the above equation, we derive the recurrence relation in 
(5.3.14). □ 

Theorem 5 .3 .3. For 2 < i < n and m = 1,2,..., 


»(. m) = 
r*t : n 


t (-l) r " n+, “ 1 (;)( r n _).)/xW. (5-3.15) 


r=n—i + 1 


Proof. By considering the expression of fi\ m) n in (5.2.8) and writing the 
term «' _1 in the integrand as {1 — (1 — «)}'“' and then expanding it binomi- 
ally in powers of 1 - u, we get 


“ “ r "‘ dU 


1-1 


Al! 


5/ ^ * ! (« -!-*)!(«- i)\(n -i + s + 1) 


r 1 ! : n -i + s + 1 ■ 


Upon rewriting this equation, we derive the recurrence relation in (5.3.15). 


□ 


From Theorem 5.3.1, for even values of n, by setting i = n/2 we obtain 

+ M ( $: J = M ( ™/2:n-|- (5-3.16) 

For the special case when m = 1, Eq. (5.3.16) gives the relation 

liP-n/2+Un + Mn/2:n} = M n /2:n-l> 


that is, 


2 ( /2+lxn + ^n/2:n)} ~ E(X n /2:n- l)> (5.3.17) 

which simply implies that the expected value of the sample median in a 
sample of even size ( n ) is exactly equal to the expected value of the sample 
median in a sample of odd size (n - 1). 

The identities that are given in the beginning of this section are quite 
straightforward. In the following theorem, we present two more identities 
due to Joshi (1973) satisfied by the single moments of order statistics which 
are interesting and simple in nature and, of course, can be useful in checking 
the computation of these moments. 
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Theorem 53.4. For n ^ 2 and m = 1,2,..., 


" 1 "1 

E -tfl = E 


i- i 


i-i 


and 


" 1 


E — TTMT’ = E JMiT , 1 

i-i " “ 1 + 1 i-i 1 


(5.3.18) 


(5.3.19) 


Proof. From Eq. (5.2.8), we have 

E TM?S- tfij/V-WlV-'o -«)•-' 

, = 1 * 

- /‘{F-'(ii)fi(l - (1 -«)’}<(«. (5.3.20) 

■'0 w 


Now upon using the identity 

1 - (1 - u) n = u Y, (1 - u)- 1 
i-i 

in Eq. (5.3.20) and simplifying the resulting expression, we derive the identity 
in (5.3.18). 

Similarly, from Eq. (5.2.8) we have 


i — - i (, l - -r‘du 

- / 0 ‘( f (j4^) I* - to- (53.21) 


1 -u" = (1 -u) £u'-‘ 

i — 1 


Upon using the identity 
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in Eq. (5.3.21) and simplifying the resulting expression, we derive the identity 
in (5.3.19). □ 


The triangle rule established in Theorem 5.3.1 for the single moments of 
order statistics is extended in the following theorem for the product moments 
of order statistics. 


Theorem 5.3.5. For 2 < i < j < n and m,, ntj = 1,2, ... , 


(,• _ + (/ - iwftjjf, + <»-; + i^PCAu. - 

(5.3.22) 


Proof. From Eq. (5.2.9), let us consider 


"Pi-l.j-l.n- 


n\ 


(/ - 2)l(j — i - l)!(/i -/)! 


XhJ 2 (Uj — u i Y ' ’(1 - Uy) n 1 dUidiij 


If {F- 1 («,)}" ,, {F- 1 (u y )}'"' 


0<U;<U ; <1 


n! 


(i- 2) !(;-« -!)!(«-))! 


XuJ 2 (u, - u,y ' \l ~ u,y '{u, + (u> - Uj) + (1 - Uj)} du<du> 


If IF" 


0<Ui<Uj< 1 


n\ 


(i-2)\(j~i- 1) !(«—;)! 

Xu| _ 1 (m> ~ Ujy _ ' _1 (l - Uj) n ~ J diij duj 


// {F- 1 (u 1 )} m '{F- 1 (u y )r 


+ ff {F 1 («,-)} m '{F l (u J )} m, u , i 2 (u j -u i y '(1 -uj) n 1 du t duj 

0 < Uj < Uj < 1 

+ // {F-\u i )} m '{F-'(u j )} m ‘ 

0<Uj <Uj<l 

Xu‘~ 2 (uj - - Uj)"- ,+l dUj du}j 

= O' - i + (; - + (»-/ + 


The last equality follows by using Eq. (5.2.9) and simplifying the resulting 
expression. □ 
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The recurrence relation in (5.3.22) due to Govindarajulu (1963a) shows that it 
is enough to evaluate the (m„ ntj ) th moment of n - 1 suitably chosen pairs 
of order statistics, if these moments in samples of size less than n are already 
available; for example, the knowledge of for 2 ^ j ^ n) or 

for 1 <, i <, n - 1) or {/*/"+"{« for 1 ^ i <, n - 1) will suffice. The (m„ m-)t h 
moment of the remaining |"j‘j pairs of order statistics can then be 
determined by repeated use of the recurrence relation in (5.3.22). Hence, it is 
desirable to express the product moment purely in terms of the 

(m,, m ; )th moment of the pairs of order statistics just given from samples of 
size up to n. These relations, due to Srikantan (1962) and Balakrishnan, 
Bendre, and Malik (1992), which can then be utilized instead of Theorem 
5.3.5 for the recursive computation of the product moments of order statis- 
tics, are presented in the following three theorems. 

Theorem 5.3.6. For \ <, i < j <, n and /n f , = 1,2 


/-I 


= y 


£ ( 

i i=n ~j +r+ 1 


1) 


s+n~i- 


:i )( s v-7 1 }(;)<«*..■ 

(5.3.23) 


Proof. By considering the expression of ” i> > n Eq- (5.2.9) and ex- 
panding the term (v - in the integrand binomially in powers of u 

and v, we get 


r*ij:n 


X // (F~ i (u i )} mi {F~ l (u j )} mi u i ~ i+r u J j~ i ~ i ~ r (l - u l Y~ i du i du i 

0<M,<My<l 


5 ( ^ 1 ( i - 1 )!(« ~i)Kr - 0 KJ ~ r - l)\ 

X // {/’- 1 ( M| .)} m< {F- 1 ( Uj .)}%r 1 «i' r " 1 ( 1 - d Ui dUj. 

0 <Wj <Uj<l 

Now writing the term uj~ r ~ l in the integrand as (1 - (1 - u / )} / ~ r ~ 1 and 
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then expanding it binomially in powers of 1 - Uj, we get 

j-lj-r-l n \ 

tfny = E E (-i) s+r " 

r=i s=0 


(i - l)\(n - j)\(r - i)\s\U - r - \ - s)\ 


X jj {F l («,)} m, {F \u j )} m 'u r i '(1 - Uj ) n 1 5 dU'duj 
0<«, <«,< 1 


j-1 j-r-1 

= E E 

r=i s=0 




n\(r - l)!(n - j + s)! 


(i - l)!(n — y)!(r - i)\s\(j - r - 1 - s)\ 
X(n -j+s + r+l)\ 


Upon rewriting the above equation, we derive the recurrence relation in 

(5.3.23). □ 


By proceeding on similar lines, we can prove the following two theorems. 
Theorem 5 .3.7. For 1 < i < j < n and /n ; , m ] = 1,2,..., 


= y 

La 


1 - 1 n 

E (-i) 

r =j — i s~n~j + r+ 1 


n —s — i + 1 


(5.3.24) 

□ 


Theorem 5.3.8. For \ < i < j < n and m,-, ntj = 1, 2, ... , 


/ : n 


«— i rt 


e e (-irr- 


r=j~i s = i + r 




fl 

S 


s-r,s : 3* 


(5.3.25) 


□ 


In the following theorem, we present a recurrence relation similar to the 
one in (5.3.22) satisfied by the covariances of order statistics established by 
Balakrishnan (1989). 


Theorem 5.3.9. For 2 < i < j < n, 

0 - IK/:* + O' - + (n-j+ l)(T i _ l] ._ Vn 

= -«-I + (/*«-! :«-I ~ M,-I:J(M/-I:„-I ~Mj: n )}- (5.3.26) 
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Proof. From Theorem 5.3.5 we have, for 2 <, i < j <, n, 

(«' - 1 + U ~ + (n~j+ l)<»i_, f y_, :il 

= fHTi-lJ-l-.n-l + n Pi- Um-lPj-l-.m-l ~ (* " 

“O' ~ ~ ( n ~j + l)M(-l:nMy-l:n 

= no i-l,j-l:n-l + :„-I 

-( n -j+ — /*/:■) 

1 , / — 1 : n — 1 — 1 : n — 1 My — 1 : n — 1 — 1 : n — 1 : n 

— «M/_| -M /:n } 
upon using (5.3.13). □ 

A warning has to be issued here that if any one of Theorems 5.3.1-5.3.3 is 
used in the computation of the single moments of order statistics, then the 
identity in (5.3.3) should not be employed to check these computations. This 
is so because the identity in (5.3.3) will be automatically satisfied if any of 
Theorems 5.3.1-5.3.3 is used, as noted originally by Balakrishnan and Malik 
(1986). In order to illustrate this point, we set i = 1, / = 2, . . . , i = n - 1 in 
Theorem 5.3.2 and add the resulting n - 1 equations to obtain 

eWj - (?k”i + E(-ir‘(;) i:'(-i)'( r 7 'k'J 

+ ( - i) n " 1 "e( - 1) 5 (" 7 1 ( 5 - 3 - 27 ) 

j-0 v ' 

By using the combinatorial identities 

eVdIV)- 0 and E 2 (-1)’(" J M -(-I)’ 

* = 0 V ' i-o 

in Eq. (5.3.27), we obtain 


n — 1 

L ,,(">) _ H„(m) _ »( m ) 

Pi:n n Pl:l Pn-.n’ 

i- 1 

which simply implies that the identity in (5.3.3) will be automatically satisfied. 

A similar warning has to be issued that if any one of Theorems 5.3.5-5.3.8 
is used in the computation of the product moments of order statistics, then 
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the identity in (5.3.4) should not be employed to check these computations as 
it will be automatically satisfied. 

In addition to the results presented above, several more identities and 
recurrence relations are available for single as well as product moments of 
order statistics. A compendium of all these results can be found in the recent 
monograph by Arnold and Balakrishnan (1989). By a systematic recursive 
usage of many of these results, they have given the maximum number of 
single and product moments to be evaluated for the computation of all 
means, variances, and covariances of order statistics in a sample of size n 
when these quantities are available in samples of sizes up to n — 1. This 
result is presented in the following theorem and interested readers may refer 
to Arnold and Balakrishnan (1989) for a proof. 

Theorem 5.3.10. In order to compute all means, variances, and covari- 
ances of order statistics in a sample of size n from an arbitrary distribution, 
when these quantities are all available in samples of sizes up to n - 1, we 
have to evaluate at most two single moments and ( n - 2)/2 product mo- 
ments if n is even, and at most two single moments and (n - l)/2 product 
moments if n is odd. 

If the population density is symmetric around zero, as mentioned earlier in 
Section 2.4, we have 


y 4 -y 

A i:n A n-i + l:n 


and 


(X i:n ,X j:n )<L(-X n _ j+l:n ,-X n _ i+l:n ). 


These results reduce the amount of Computation involved in the evaluation 
of moments of order statistics from a symmetric population. For example, 
from Theorem 5.3.1, we get in this case 


An/2) : n = / 2 > : „ - 1 . for n even and m even (5 .3 .28) 

and 

An/2y.n-i = °> for n even and m odd. (5.3.29) 


These two results are seen to decrease immediately the number of single 
moments to be evaluated in a sample of size n that is given in Theorem 
5.3.10 by 1. By using some similar arguments, Joshi (1971), David (1981), and 
Arnold and Balakrishnan (1989) have determined the maximum number of 
single and product moments to be evaluated in a sample of size n from a 
symmetric population and their result is presented in the following theorem 
without a proof. 
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Theorem 53.11. In order to compute all means, variances, and covari- 
ances of order statistics in a sample of size n from a symmetric distribution, 
when these quantities are all available in samples of sizes up to n - 1, we 
have to evaluate at most one single moment if n is even, and at most one 
single moment and ( n - l)/2 product moments if n is odd. 


5.4. UNIVERSAL BOUNDS 

In this section, we derive universal bounds for the expected value of the 
largest order statistic X„.„, a general order statistic A r ( .„, and the spacing 

First, let us consider (without loss of generality) an arbitrary population 
with mean 0 and variance 1; that is, 

f l F~'(u)du- 0 and f l [F~ l (u)} 2 du = 1. (5.4.1) 

J o J o 

Then, by writing the expected value of the largest order statistic as 

Mn:n = f 1 F~ , (u)(rtu n ~ l — A) du (5.4.2) 

J Q 

and using the Cauchy-Schwarz inequality, we obtain 

/ 2 , J/2 

'‘■" £ {^' 2A + A / ' <5 ' 4 ' 3) 
By noting that the RHS of (5.4.3) is minimum when A = 1, we simply obtain 

( n - 1 ) 

u„ < — T77. (5.4.4) 

(2 n - 1) ,/2 

This result was originally derived by Hartley and David (1954) and Gumbel 
(1954) and was discussed further by Moriguti (1951, 1954). Further, from 
(5.4.2) we realize that the bound in (5.4.4) is attained when and only when 


F ‘(m) = c(nu" 1 - 1), 0<u<l. 

The constant of proportionality c is determined from Eq. (5.4.1) to be 
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c = (2 n - 1 ) i/2 /(n - 1), which then yields 


F-'(u) 


(2 n - 1) 1/2 

(«-l) 


( nu n ~ l 


- 1 ), 


0 < u < 1. 


(5.4.5) 


Hence, after noting that F~\u) in (5.4.5) is a monotonic increasing function 
in u, we obtain the distribution for which the bound in (5.4.4) is attained 
to be 


F(x)= - 


1/ (n — 1) 


!/(«-!) 


(2n - 1) 


1/2 


X + l 


( 2 n - 1 ) 
(n-1) 


1/2 


<x < (2n - 1) I/2 . (5.4.6) 


Similarly, by considering an arbitrary population with mean 0 and variance 
1 and by writing the expected value of the z'th order statistic as 

■ /o V "(" > {s( i r/, + i )"'" (1 - u) " ‘ du ~ A } du (5 - 4 - 7) 

and using the Cauchy-Schwarz inequality, we obtain 


I J ^ 


B(2i -1,2 n- 2 i + 1) 
{B(i,n - i + l)} 2 


1/2 


- 2A + A 2 


(5.4.8) 


By noting that the RHS of (5.4.8) is minimum when A = 1, we simply derive 
Ludwig’s (1959) bound that 

^ { B (2i - l,2n - 2 i + 1) - {B(i,n - i + l)} 2 } 1/2 /B(i, n -i + 1), 

(5.4.9) 


where B{ - , ■ ) is the complete beta function as defined in (2.2.18). We may 
note here that the bound in (5.4.9), for the case when i = n, becomes the 
same as the bound in (5.4.4). Further, by denoting 

1 

B(i, n — i + 1) 


S(u) = 


u'-'(l - 1, u e (0, 1), (5.4.10) 
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we obtain 


g '( u ) = — ~ 1)«'“ 2 (1 - - in - <)« ,_, (1 - 

B(t, n - i + 1) 

- ^-r— -r-w' 2 (1 - - 1) - (n - 1)«}, u e (0, 1). 

B(i,n - i + 1) 

(5.4.11) 


It is quite clear from the expression of g'(u) in (5.4.11) that the function g(u) 
in (5.4.10) becomes a monotonic function in u only when i = 1 or i = n. 
Hence, the bound in (5.4.9) is attainable only when i = 1 or i = n. 

Next, we shall derive the universal bound for the expected value of the 
spacing W i ;j:n = X j:n - X i:n , 1 < i <j < n. For an arbitrary population with 
mean 0 and variance 1, by writing the expected value of W i j n as 




1 


B(i,n - i + 1) 


u' *(1 - u) - A> du 


and using the Cauchy-Schwarz inequality, we obtain 


(B(2j -1,2 n- 2) + 1) B(2i -1,2 n- 2i + 1) 
E(W iJ:n )zl-^ + 


{B(j,n - j + 1)}' 


[B(i,n - i + 1)}' 


+/ ~ l,2n - i -j + 1) + 

B(i,n - i + l)B(j,n -j + 1) 


1/2 


. (5.4.12) 


By noting that the RHS of (5.4.12) is minimum when A = 0, we obtain 
Ludwig’s (1959) bound that 


E{W u , n ) ^ 


B(2j -1,2 n- 2 j + 1) B(2i -1,2 n- 2i + 1) 

{B(j,n-j + l)} 2 + [B(i, n — i + l)} 2 

B(i +j - l,2n - i -j + 1) ) 1/Z 
2 B(i,n - i + l)B(j,n -j + 1) J 


(5.4.13) 


By setting ;=/»-/ + 1, in particular, we deduce from (5.4.13) the universal 
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bound for the expected value of the *th quasirange W rn = X n _ i+x .„ - X,, n , 
1 <, i < in + l)/2, to be 

E{Win) * + l) {H2i ~ 1 ’ 2n ~ 2i + l> >- 

(5.4.14) 


For the special case when i = 1, we deduce the universal bound for the 
expected value of the sample range W n = X n:n - X l:n to be 


E{W n ) < 



(5.4.15) 


The bound in (5.4.13) can be shown to be attained only when i = 1 and 
j = n. 

The bounds derived in (5.4.4), (5.4.9), and (5.4.13) can be improved for the 
case when the population distribution is symmetric about zero. For this 
purpose, we use the relation in (2.4.13) and write the expected value of 
the largest order statistic X n . n from a symmetric population with mean 0 and 
variance 1 as 


V L n-.n = lC F '(“){«“" 1 - n(l - u) n 1 - A} rfu. (5.4.16) 
J o 

By applying the Cauchy-Schwarz inequality to the above integral, we immedi- 
ately obtain 


1 I 2 n 2 

/i " :n “ 2 1 2/1-1 “ 2 B ( n ’ n ) + y 


1/2 


(5.4.17) 


By noting that the RHS of (5.4.17) is minimum when A = 0, we obtain 

1/2 




{ 2 ( 2 n - 1 )} 


1/2 


1 - 


1 


[2n - 2’' 
l n - 1 , 


(5.4.18) 


It is of interest to mention here that the bound in (5.4.18) is exactly half of 
the bound in (5.4.15) and this could be explained through the fact that 
/i n:n = jE(W„) for a population symmetric about zero. From (5.4.16) we note 
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that the bound in (5.4.18) is attained when and only when 

F~ l (u) = c{nu"~ l - n( 1 - u)' 1 " 1 }, 0 < u < 1. 

The constant of proportionality c is determined from Eq. (5.4.1) to be 



which then yields 

1/2 

{ u '’" 1 - (1 - m )"" 1 }, 0 < m < 1 . 

(5.4.19) 

Hence, after noting that F~\u ) in (5.4.19) is a monotonic increasing function 
in w, we conclude that the bound in (5.4.18) is attainable and is attained for a 
symmetric population with its inverse cumulative distribution function as 
given in (5.4.19). 

Similarly, by using the relation in (2.4.13) and writing the expected value 
of the ith order statistic X i:n from a symmetric population with mean 0 and 
variance 1 as 





B(i,n - i + 1) 


^-'(1 -u)‘ '~-k}du (5.4.20) 
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By noting that the RHS of (5.4.21) is minimum when A = 0, we obtain 


I Mi:J < 


n — i + 1) 


{B(2i - 1,2 n - 2 i + 1) -B{n,n)} l/Z . 


(5.4.22) 


We note that the bound in (5.4.22), for the case when i = n, simply reduces 
to the bound in (5.4.18). Further, we observe that the bound in (5.4.22) is 
exactly half of the bound in (5.4.14), which could be explained through the 
fact that \n i:n \ = jE(W i:n ) for a population symmetric about zero. It may be 
shown in this case that the bound in (5.4.22) is attainable only when i = 1 or 
i = n. 

Finally, by using the relation in (2.4.13) we may write the expected value of 
the spacing W iJ:n = X ]: „ - X i: „, 1 <i<j <n, from a symmetric popula- 
tion with mean 0 and variance 1 as 




1 


B(j,n -j + 1) 
1 

B(i, n - i + 1) 
1 


+ 


«""'( 1 - u) 


u ‘~ i ( i - u y 


«"-'(! -«)' '-A \du. (5.4.23) 


/- 1 


B(i,n —i + l) 


By applying the Cauchy-Schwarz inequality to the above integral, we immedi- 
ately obtain 


E ( W iJ:n )<-{2 


If B(2j-l,2n-2j + l) B(2i - l,2n - 2i + 1) 


+ 2 - 


-2 


-4 


{B(i, n — i + l)} 2 
B(n,n) 


{ B(j,n - j + l)} 2 
B(n,n) 

[B(j,n -j + l)} 2 {B(i, n — i + l)} 2 

B(i + j - l,2/i - i - j + 1) 

B(i,n - i + 1 )B(j,n -j+ 1) 


+ 4- 


B(n - i + j,n - j + i) 
B{i, n - i + \)B(j,n - j + 1) 


1/2 


+ A 2 . (5.4.24) 
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By noting that the RHS of (5.4.24) is minimum when A = 0, we obtain 


EWj:n) * 


1 


(B(j,n -j + 1)} 
1 


+ ■ 


{B(i,n - i + 1)} 
2 


2(B(2j -1,2 n - 2 j + 1) - B(n, «)} 

2(B(2i - l,2n - 2 i + 1) -B(n,n)} 


B(i,n - i + 1 )B(j,n - j + 1) 


(B(i +j-l,2n-i-j + 1) 


-B(n - i + j,n - j + i )} 


1/2 


. (5.4.25) 


It is important to mention here that the bound for the expected value of the 
ith quasirange deduced from (5.4.25) is identical to the one derived in 
(5.4.14) and thus no improvement is achieved in the case of symmetric 
distributions. 


Table 5.4.1. Universal Bounds for E(X„ :m ) and Exact Values for the 
Uniform! - V*3 , +V*3 ) and NormaKO, 1) Populations 





Exact value 

Exact value 


Bound in 

Bound in 

for the 

for the 

n 

(5.4.4) 

(5.4.18) 

Uniform! — ^3 , + ^3 ) 

NormaKO, 1) 

5 

1.33333 

1.17006 

1.15470 

1.16296 

10 

2.06474 

1.62220 

1.41713 

1.53875 

15 

2.59973 

1.96960 

1.51554 

1.73591 

20 

3.04243 

2.26455 

1.56709 

1.86747 

25 

3.42857 

2.52538 

1.59882 

1.96531 

30 

3.77548 

2.76172 

1.62031 

2.04276 

35 

4.09312 

2.97940 

1.63583 

2.10661 

40 

4.38784 

3.18223 

1.64756 

2.16078 

45 

4.66399 

3.37289 

1.65674 

2.20772 

50 

4.92469 

3.55335 

1.66413 

2.24907 

55 

5.17226 

3.72507 

1.67019 

2.28598 

60 

5.40852 

3.88922 

1.67526 

2.31928 

65 

5.63489 

4.04672 

1.67956 

2.34958 

70 

5.85250 

4.19832 

1.68326 

2.37736 

75 

6.06232 

4.34463 

1.68647 

2.40299 

80 

6.26511 

4.48618 

1.68928 

2.42677 

85 

6.46154 

4.62339 

1.69177 

2.44894 

90 

6.65217 

4.75665 

1.69398 

2.46970 

95 

6.83749 

4.88627 

1.69597 

2.48920 

100 

7.01792 

5.01255 

1.69775 

2.50759 
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In order to give an idea about the sharpness of these bounds, we have 
presented in Table 5.4.1 the bounds for the expected value of the largest 
order statistic calculated from (5.4.4) and (5.4.18) for sample size n = 5(5)100. 
Also given in Table 5.4.1 are the exact values of E(X n n ) for the 
Uniform(- /3 , + /3 ) and Normal(0, 1) populations. The exact values of 
E(X n:n ) for the standard normal distribution have been taken from the 
tables of Harter (1970) and for the Uniform(-v^3 , + /3 ) distribution they 
have been calculated from the exact formula given by ifl2[n/(n + 1) - \]. It 
is clear from this table that the bound in (5.4.18) based on a symmetric 
distribution gives a considerable improvement over the bound in (5.4.4). It is 
also clear from this table that both these bounds become less sharp as n 
increases. Note, however, that in the case of Uniform(-v^3 , + /3 ) distribu- 
tion even the bound in (5.4.18) is not useful when n > 12 as it is more than 
+ v^3 . Interested readers may refer to David (1981) for some improvements 
on these bounds when the population distribution has a finite support, such 
as the case here. 

Similarly, we have presented in Table 4.5.2 the bounds for the expected 
value of the ith order statistic calculated from (5.4.9) and (5.4.22) for sample 
size n = 10 and 20 and i = tt/2 + l(l)n. We have also included in this table 
the exact values of E{X i n ) for the Uniform(- /3 , + /3 ) and Normal(0, 1) 
populations. The necessary values of E(X i:n ) for the standard normal 


Table 5.4.2. Universal Bounds for E(X i:n ) and Exact Values for the 
Uniform! — ^3, + y/3 ) and NormaKO, 1) Populations when n = 10 and 20 


n 

i 

Bound in 
(5.4.9) 

Bound in 
(5.4.22) 

Exact value 
for the 

Uniform( -^3, +\/3) 

Exact value 
for the 
NormaKO, 1) 

10 

6 

0.95370 

0.30900 

0.15746 

0.12267 


7 

1.00024 

0.78634 

0.47238 

0.37576 


8 

1.10865 

1.02196 

0.78730 

0.65606 


9 

1.33656 

1.17848 

1.10221 

1.00136 


10 

2.06475 

1.62220 

1.41713 

1.53875 

20 

11 

1.26752 

0.25528 

0.08248 

0.06200 


12 

1.27847 

0.69886 

0.24744 

0.18696 


13 

1.30117 

0.98901 

0.41239 

0.31493 


14 

1.33741 

1.13469 

0.57735 

0.44833 


15 

1.39051 

1.20297 

0.74231 

0.59030 


16 

1.46657 

1.25428 

0.90726 

0.74538 


17 

1.57747 

1.32063 

1.07222 

0.92098 


18 

1.75013 

1.42530 

1.23718 

1.13095 


19 

2.06562 

1.62278 

1.40214 

1.40760 


20 

3.04243 

2.26455 

1.56709 

1.86748 
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distribution have been taken from the tables of Harter (1970) and for the 
Uniform(-\/3 , +\/3) distribution they have been computed from the exact 
formula given by {vL\i/{n + 1) - |]. It can be noted from Table 4.5.2 that 
the bound in (5.4.22) based on a symmetric distribution gives a good improve- 
ment over the bound in (5.4.9) for all /. Once again, we observe that 
increasing values of n make both these bounds less sharp. 

Several improvements and generalizations of these bounds have been 
presented by Arnold and Balakrishnan (1989). Some other methods of 
deriving bounds for expected values of order statistics have been discussed in 
detail by David (1981) and Arnold and Balakrishnan (1989). 


5.5. SERIES APPROXIMATIONS 

As pointed out in (1.1.3) and (2.4.2), we have 

Xitn-F-'W:*) a "d (Xin,X J:n )±(F- l (U i:n ),F-'(U J:n )). (5.5.1) 

Upon expanding F _1 (^ ;n ) in a Taylor series around the point E(U i n ) = 
i/(n + 1) = p„ we get a series expansion for X i n from (5.5.1) as 

= F-\Pi) + F- lW (Pm :n -Pi) + \F- li2 \Pi){V hn -pf 

+ ^' 1(3) (P/)(^:„ -Pif + JiF- lW (PiM:n ~ Pi)' + ”• , 

(5.5.2) 

where F~ 1(1) ( p f ), F~ 1(2> ( p,), F~ 1(3) ( p,), F~ 1(4) ( p,), ... are the successive 
derivatives of F~ l (u ) evaluated at u - p,. 

Now by taking expectation on both sides of (5.5.2) and using the expres- 
sions of the central moments of uniform order statistics derived in Section 4.7 
[however, written in inverse powers of n + 2 by David and Johnson (1954) 
for computational ease and algebraic simplicity], we derive 


P i:n = F-'( Pi ) + 


PiQi 

2(n + 2 ) 


F- ,(2) (p,.) 


+ 


Pi<li 


+ 


( n + 2 Y 
PiQi 


(n + 2y 


y( Qi~Pi)F 1 ( 3 ) (p,)+ -PiQiF~ m (Pi) 

- j(4, -p,)F- 1(3) (p,) + ^ {(<?, - P,) 2 - PiQi) F~ 1(4) (p,) 


+ ^PiQiiQi ~ Pi)F~ liS) (Pi) + ^ %Pf<lfF m (Pi ) 


, (5.5.3) 


where <?, = !- p, -(«—/+ 1 )/(n + 1). 
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Similarly, by taking expectation on both sides of the series expansion for 
Xf. n obtained from (5.5.2) and then subtracting from it the approximation of 
p.] :n obtained from Eq. (5.5.3), we derive an approximate formula for the 
variance of X, . „ as 

»u:. - ^r(f“ lll (ft)) 2 + -p,)F-“ n (p,)F-m(p,) 

n + l (n + 2) 

+PA,[F' m KP,)F- im (P,) + ^- 1(2) U)} 2 ] 

+ 7 [ - 2( 9i - Pi)F - 1(1) ( P,) F ~ 1(2) ( Pi) 

(n + 2) 

+ {(9/-P 1 ) 2 -P/9 1 | 2F“ I(I) ( Pi)F~ 1(3) ( p ( ) + |{ J F- 1(2 >(p i )} 2 
+A9 i (rPi)j^- ,(,1 (P i )f ,<4) (P i ) + 3F-«W( Pi )f- ,<3, (Pi)} 

+ ^p,V F~ ,(I) ( Pi )F~ I(5) ( Pi ) + 2F~ ,(2) ( Pi)F~ t(4) ( p t ) 

+ j{F~ H3) (P,)} 2 • (5-5-4) 

Finally, by proceeding similarly and taking expectation on both sides of the 
series expansion for X t . „ X ] . n obtained from (5.5.2) and then subtracting 
from it the approximation for p. l n p. j:n obtained from (5.5.3), we derive an 
approximate formula for the covariance of X i:n and X j:n as 

-~zF- Pi )F~ 1(1) ( Pj ) + - P f 2 - L, - Pi )F~ « 2 >( Pi )F~ Pj ) 
t\ -r l (n + l) 

+ (<?/ ~ Pj)F-^\ Pi )F-^\ Pj ) + j^,F" 1(3) (P,)F" 1(1) (P,) 

+ ~P j qjF- lm (p i )F-'<\p J ) + ^MjF-WmF-WIpj) 

+ 7~" 3 - (9, - Pi) F ~ U2) ( Pi)F~ 1(1) ( Pj) 

(« + 2 ) 

-(qj ~ Pj)F~ n '\Pi)F~ H2 \pj) + {(<?, - Pi ) 2 -p/9«}F- 1(3) (P/)F _1(1) (p y ) 

+ {(<7, “ P/) 2 - Pjq,}F" 1(1) ( Pj)F~ 1(3) ( p y ) 

/ 3 1 ^ 

+ ~P*K«y — Pj) + " 2 P/9y) F ~ I<: ^ ) (Pi)F“ 1(: 2 \pj) 
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+ - Pi)F m (Pi)F im (Pj) + ^PjQjiQj - Pj)F im (Pi)F~ m (Pj ) 


+ {p,<7,(<7, -Pi) + ^PiQiiqj ~ Pj^F'^XpJF-Wipj) 

+ | Pi<lj(<lj ~ Pj) + ^PjQjiQt ~ Pi)^F~ K 2 \Pi)F~ 1 ( 3 \Pj) 

+ \pMF-^( Pi )F-^(Pj) + \pjqjF-^\ Pi )F-^\ Pj ) 

+ jpfqfl j F-W>(p i )F- v a(pj) + ~PiPj<]jF~ l ( 2 \ Pj)F~ 1(4) ( pj) 
+ ~{ 2 Phi + IPiPjWjjF-^iP^F-^Xpj) . 


(5.5.5) 


The evaluation of the derivatives of F~\u ) is rather straightforward in 
case of distributions with F~ l (u ) being available explicitly. For example, for 
the logistic distribution with pdf 

f(x) = e~ x /(l + e~ x ) 2 , -» <x < oo, 

and cdf 

F(x) = 1/(1 + e~ x ), -oo <x < oo, 

we have 

x = F~ l (u ) = log u - log(l - u). 


We then obtain 


1 1 

F~ l(l) (u) = - + , 

u 1 - u 

1 1 

F -1(2) (u) = --J + 2, 

V ' M 2 (1 -«) 

(1 1 ) 

F- 1(3) (u) = 2 -3 + 


(l-) 3 /’ 

1 

(1 - u ) 5 

F -1(6) (m) = 120/ — -^ + 1 


F _1(5) (u) = 24 — + 


1 


(1-u) 


6 f ’ 


and so on. 



SERIES APPROXIMATIONS 


131 


Fortunately, the evaluation of the derivatives of F~\u) is not difficult 
even in case of distributions with F~'(u) not existing in an explicit form. In 
this case, by noting that 


F~ Hl \u) 



dx 

du 


1 

( du /dx ) 


1 

TTo 


1 

f(F~ l (u))' 

(5.5.6) 


which is simply the reciprocal of the pdf of the population evaluated at 
F~ l (u), we may derive the higher-order derivatives of F~\u) without great 
difficulty by successively differentiating the expression of F~ l(i Ku) in (5.5.6). 
We shall illustrate this by considering the standard normal distribution. In 
this case, by making use of the property that ( d/dx)f(x ) = -xf(x), we 
obtain 


F-« 2 \u)=F-\u)/{f(F- l (u))f, 

F-™(u) = {l + 2{F-\u)) 2 }/{f{F-\u)))\ 

F~ m (u) = + 6{ J P->(“)} 2 }/{/(^- 1 («))} 4 , 

f- 1(5) (u) = {7 + 46 {F-'(h)} 2 + 24{F -i (m)} 4 }/{/(F~ 1 (m))} 5 , 

F~ m (u) = F-\u){m + 326{F-\u)} 2 + 1201^- 1 ( u)} 4 }/{/( 1 ( M ))} 6 , 


and so on. 

David and Johnson (1954) have given similar series approximations for the 
first four cumulants and cross-cumulants of order statistics. As mentioned 
earlier, these series approximations are given in inverse powers of n + 2 
merely for the simplicity and computational ease. It should be mentioned 
here that Clark and Williams (1958) have developed series approximations 
similar to those of David and Johnson (1954) by making use of the exact 
expressions of the central moments of uniform order statistics where the fcth 
central moment is of order {(n + 2 Xn + 3) • • • (n + k))~ l instead of in 
inverse powers of (n + 2). These developments and also some other methods 
of approximation have been discussed in great detail by David (1981) and 
Arnold and Balakrishnan (1989). 
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EXERCISES 


1. For l = 1, 2 n - i and m ^ 1, prove that 


(n - ifvn = e 

r = 0 ' ' 

where rt (m) = 1 when m = 0 and = n(n - 1) • • • (n - m + 1) when 
m ^ 1. 

Similarly, for l = 1, 2, . . . , i - 1 and m ^ 1, prove that 

(«' - = E ('-/-0 ( '" n+r >- r) („i r k-^- 

r=n-t 

Using these relations, deduce the results of Srikantan (1962) presented in 
Theorems 5.3.2 and 5.3.3. 


2. For \ <, i <1 < n and m ^ 1, show that 


("Mi? - "e(' + r ! )(" yi 7 


(Sillitto, 1964) 


3. For k + l <, n - 1 and m ^ 1, show that 


"e (i - i) <w (« - 0 <o MT2 = 

_ i. j. i ' * 


similarly, for k + l ^ n - 2 show that 


n — 1 — / /i — I 


E E («“ ~}) U) l i i,j:n - kwifc + l + 2 W + l,*+2:*+/+2‘ 

i-=L + l y— i + l 

(Downton, 1966) 

4. For any function h for which E(h(X )) exists, prove that for i = 
1, 2 n, 

£(*(*.:„)) ' ( 7) £ (~l) r 7^7 T-„ 


n -j + r\‘ 

, l ~j , 


0 £j <, i - 1, 
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and 




£{*(*,:„)} = (7 ) jE( - 

\ i + r ) 

0 <j < n - i. 
(Krishnaiah and Rizvi, 1966) 

5. For any continuous population with cdf F(x), show that 

i 


E{F(X i n )X j n ) = ^~[Fj + i.„ + i, 1 < i <j ^ n. 


and 


E{Xi-.n F { X j : n)} = Fi:„ ~~ j Mi : n + I > 1 < I <j <H. 


n + 1 


(Govindarajulu, 1968) 


6. Prove that 

M/7«= E E (-!)"( 2 < i < n, m > 1, 


and 


n r 


/47n = E E (-l) i+I (r)(j)M < i'?i-r +J > 1 < i < n, m > 1, 

r=i 5 = 0, 1 

where the summation from s = 0, 1 to r denotes the sum from 0 to r 
when r < n and from 1 to r when r = n. 

(Young, 1970) 

7. Let <f> be a function such that </>, .„ = E{<j>{X in )) exists for every i < n 
for every n. A doubly infinite matrix A with elements a rs will be called 
an admissible matrix, for a fixed i and n, if 

8a( u ) = E E - u) s ~ n+ ‘ = 1, mg [0,1], 

r= 0 i=0 

If A is admissible for i and n, then show that 

“ “ B(r + 1,5 + 1) ( 

* i:n rhos-o 0 " B(i,n-i+ 


(Arnold, 1977) 
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8. For r > 1 and m ^ 1, show that 


LtfZAi + r- l) (r> 

i=i 

and 


E tfl/in-i + rf' 

i = i 


1 

(n + r- l) (r_1) 


1 

(n + r- l) (r_1) 



i + r -2 
r- 1 


i+r - 2 
r- 1 


)mV ”]/i 

y$/i- 


Note that these results generalize Joshi’s (1973) identities presented in 
Theorem 5.3.4. 

(Balakrishnan and Malik, 1985) 


9. Prove the identity 


E 0 + 0 " '/,:«(*) = E (”)/i:i(*)*' foran y '• 

i=i i=i v ' 


Let A and E be the difference and the shift operators with common 
difference 1 acting on functions of y (independent of jc). Then, deduce 
the operator equality 


E/ ( ,„(jc)£"-' = E(.)/ 1:i (*)A'- 1 . 

i -1 i * 1 V 

Similarly, prove the dual operator equality 

i=l i” 1 ' ’ 


For the choice of the function T(y) = l/(n - y) and T(y) = 1/ 
(n + r - 1 - yY r \ show then that the above two operator equalities 
yield Joshi’s (1973) identities presented in Theorem 5.3.4 and the gener- 
alized identities given in Exercise 8, respectively. Can you derive some 
simple identities by making some other choices for the function T(y)? 

(Balasubramanian, Balakrishnan, and Malik, 1992) 

10. Denoting the mean of the range W n by <o n = fi n .„ - the mean of 
the ith quasi-range W i n by (a t , n = M n -i+i:n “ Mi ;n > an ^ niean 
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difference between O' + l)th and ith order statistics by = M, + i :n ~ 



establish the following relations: 


(a) 

(n - l)co n = w 2 n , n 

> 3. 

(b) 

1 

^ n ^ n — I 4" ( X 1 ; n Xn — 1 : n) > 

n 

n > 3. 

(c) | 

;) E ( _1 ) r+1 (r) a, n -/ + r = Xi-.n + Xn-i:n> 

1 < / < n - 1 

(d) 

E(-i)'(?)» r , 

r = 2 

n > 3. 

(e) 

"- 1 , , 

2a > n = Y ( ~ 1)( r ) w r, for odd values of n . 

r = 2 V ' 

(f) 

nXi-l.n-\ -(«-« + = iXi-.nf 

2 < / < n — 1 . 


, , " „r( k \ i»-i + rr 

( g ) Xi-.n .(k) (rj^ _ k + r yr)Xi-k:n-k + r> 

k < i ~ 1. 

( h ) 1 </<[—— ]. 


(Sillitto, 1951; Cadwell, 1953; Romanovsky, 1933; Govindarajulu, 1963a) 

11. Prove that if Theorem 5.3.5 (or any of Theorems 5.3.6-5.3.8) is used in 
the computation of the product moments of order statistics, then the 
identity in (5.3.4) will be automatically satisfied and hence it should not 
be employed to check the computations. 

(Balakrishnan and Malik, 1986) 

12. For 1 < i < n — 1 and 1 < k < n - i, show that 
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Then, deduce the following results: 


(a) 


n-k + \ , • % * + l 


n-j 

n-k- 



n 



Ml:*Ml: 


n-k> 


n i 

(b) E Mi, j;n + E/ir,i + l:n = "Ml : iMi : „ - 1 > 

j— i + 1 r*= 1 


(c) For distributions symmetric about 0, 


n 


I] Mi, j.n 

S = i+1 


E Mr,i + 

r= 1 


1 : n 


= o, 


1 < i < n - 1 , 


and hence, for even values of n (say n = 2m) 


E^r.m + l:2m 


r= 1 


j = i + 1 


r= 1 


Idl 52 °i,i:n E°r.i + l:n — |*Ml:l E Mr : n | ( Mi' + 1 :n Mi :n)> 

1 £ i ^ n - 1. 
(Joshi and Balakrishnan, 1982) 


13. Prove that 


(« - 0(" - i - 1) E {(')/(" r 2 )} i Mn— r— l,n— j:n 
r»=0 ' ' ' s-0 

= n(n - l)M„-/-i, 1 < i £ « - 2, 
and 

n-i n i n \ nl ( n \ 

E Mi,i + l:n + E I; jMl.j:/ = E I j jM;:;Ml :n-;> « ^ 3. 

i-1 /-2' ' >-l X 7 

(Joshi and Balakrishnan, 1982) 

14. If g(x) is any differentiable function such that differentiation of g(x) 
with respect to x and expectation of g(X) with respect to an absolutely 
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continuous distribution are interchangeable, then show that 

£{*'(*,:■)}=- ZE{g{X i , n )f'{X j:n )lf{X j , n )), 1 <izn, 
>=i 

where /(*) is the density function of X. 

(Seal, 1956; (Govindarajulu, 1963a) 

15. If g(x) is any twice differentiable function such that twice differentiation 
of g(x) with respect to x and expectation of giX) with respect to an 
absolutely continuous distribution are interchangeable, then show that 
for 1 < i < n 

E{g”{X i:n )) = Elg(X i:n ) i nX jLn )/f(X i:a ) 

V y=i 

v j*k > 

similarly, if g and h are differentiable functions such that differentiation 
of g(x)h(x) with respect to x and expectation of g(X)h(X) with respect 
to an absolutely continuous distribution are interchangeable, then show 
that 


E[g'(X i:n )h(Xj. n ) + g{X i:n )h’[Xj n )\ 

= - E£{«(^. n )M^: n )/'(^ n )//(^: n )}- 

* = 1 ’ 

(Govindarajulu, 1963a) 


16. Suppose X l:n <X 2:n < ••• <X n .„ are the order statistics from a 
population with cdf Fix) and pdf /( x) symmetric about 0. Further, 
suppose Y l n <Y 2 n < ••• <Y n:n are the order statistics from the 
folded population (folded at 0) with pdf pix) = 2 fix) and cdf Pi x) = 
2 Fix) - 1, x * 0. Let us denote EiX” n ) by ^ EiX r n X j:n ) by n, J:n , 
EiYi?„) by and EiY i n Yj. n ) by v, /;n . Then, prove the following two 
relations: 

(a) For 1 ^ i £ n and m > 1, 




r=0 
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(b) For 1 <, i <j <, n, 


2 V 



E.(?) y r-y+l,r-i + l:r 




-e(") 


*V-i+ l:r v j 


—r* 


(c) If e is the error committed in computing each of the p’s, show then 
that the maximum cumulative rounding error that will be committed 
in computing through the relation in (a) will also be e. 

(Govindarajulu, 1963b) 

17 . Let X u X 2 X n be n exchangeable random variables with joint cdf 

F(x u x 2 x n ); that is, F(x x , x 2 x n ) is symmetrical in x x , x 2 

x n . Then, prove that the results presented in Theorems 5.3.1-5.3.3 and 
3.3.3 continue to hold in this case as well. 

(David and Joshi, 1968) 

18 . Prove that the bound for E(W u . n ) in (5.4.13) is attainable only when 
i = 1 and j = n. What is the extremal distribution in this case? 

19. Explain why the bound for \n i:n \ in (5.4.22) is attainable only when i = 1 
or n. What is the extremal distribution in this case? 

20 . Explain why the bound for the expected value of the ith quasirange W i:n 
obtained from (5.4.25) (based on the assumption that the population is 
symmetric) is identical to the one obtained from (5.4.14) (based on an 
arbitrary population). 

21 . Derive an universal bound for the expected value of the midrange V n = 
\(X l:n + X n:n ). Discuss whether the bound is attainable, and if so, what 
is the extremal distribution? What can you say about the bound for the 
symmetric distribution case? Can you generalize the results to the ex- 
pected value of j(X i:n + X J : „), l <, i <j < n! 

22 . For the extreme-value distribution with pdf 

/( x)=e~ e *e x , -oo <jt<ao, 

work out the function F~\u) and its first six derivatives that are required 
for David-Johnson approximation discussed in Section 5.5. Using these 
expressions and the formulas in Section 5.5, find approximate values of 
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/i (;10 , <r iI: 10 , and (Tj j. w , compare these values with the exact values 
tabulated by Balakrishnan and Chan (1992), and comment on the accu- 
racy of the approximation. Incidentally, this distribution is one of the 
possible limiting distributions for the sample minimum; for details, see 
Theorem 8.3.5. 


23. For the gamma population with pdf 


f(x) 


r (p) 6 


- x x p -\ 


x > 0, p > 0, 


work out the derivatives of F~ l (u). Using these expressions and the 
formulas presented in Section 5.5, determine approximate values of p iA0 
and <r, , . , o when p = 2 and 3, and comment on the accuracy of the 
approximation by comparing these values with the corresponding exact 
values tabulated by Gupta (1960). 




CHAPTER 6 


Characterizations using 
Order Statistics 


6.1. WHO CARES? 

Reactions to characterization results are usually of two kinds. Either one 
exhibits fascination and wonderment or spits out a “who cares?” For exam- 
ple, the classical characterization result involving order statistics is as follows. 
Note that for samples from an exponential distribution X l:n has the same 
distribution as X x . , except for the rescaling, and this is true for any sample 
size n. Thus the form of the survival curve is the same for any series system 
constructed using such i.i.d. exponential components. Is the exponential 
distribution the only distribution with this property? The answer is yes. In 
fact, under mild regularity conditions, we shall see that it is enough that the 
result hold for any one value of n. The result is more than curious. It 
highlights an important consequence of the assumptions inherent in using an 
exponential model. It may well provide an irrefutable argument against an 
exponential model in some situations. Perhaps engineering experiences dic- 
tate that series systems with many components have survival curves of 
markedly different shape than do systems with few components. If so, then 
we must forego using an otherwise attractive exponential model. 

A second important use of characterizations is in the development of 
goodness-of-fit tests. The present example can be used in this manner 
(though the resulting test is not the best available). Suppose we have a large 
number of observations from a distribution F, and we wish to test whether 
the distribution F is exponential with some unknown unspecified mean 0. 
We could act as follows. Randomly split the data into thirds, relabeling for 
convenience to get 

V V 

r \ > • • • > r m> 
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where m - [n/ 3]. Now for i = 1, 2, . . . , m let Z, = 2min(^, Wj). Now ac- 
cording to our earlier result, the samples U u ...,U m and Z v ...,Z m will 
have a common distribution if and only if the original data were exponential. 
So to test our hypothesis, we merely compare the sample distribution 
functions of the IT s and the Z’s using a standard two-sample nonparametric 
test (Kolmogorov-Smimov or some other favorite). 

The general message is that useful characterization results are those which 
shed light on modeling consequences of certain distributional assumptions 
and those which have potential for development of hypothesis tests for model 
assumptions. It is true that one must show discretion in cataloging character- 
ization results. Clearly, since a distribution is determined by its values at a 
countable number of points, almost any countable number of conditions on a 
distribution should go a long way toward characterizing a distribution. Two 
key references on characterizations are Kagan, Linnik, and Rao (1973) and 
Galambos and Kotz (1978). Both contain an interesting discussion of motiva- 
tion for the studies of characterizations. The former focuses on characteriza- 
tions involving linear combinations of i.i.d. random variables, and has the 
normal and stable distributions in the spotlight. The latter, focusing on the 
exponential distribution and its close relatives, is more concerned with 
characterizations based on order statistics. It, consequently, provides a valu- 
able reference for the present chapter. 

It is clear that knowledge of the distribution F l:l completely determines 
the distributions F i:n for every i,n. It also completely determines the 
marginal and joint distributions of various linear combinations of order 
statistics. Our goal is to investigate the obverse of this coin. To whit, we ask 
to what extent does knowledge of some properties of the distribution of an 
order statistic or of some linear combinations of order statistics determine 
the parent distribution? 


6.2. THE DISTRIBUTION OF AN ORDER STATISTIC DETERMINES 
THE PARENT DISTRIBUTION 

Suppose that X lt ...,X n are i.i.d. with common distribution F. Clearly, 
knowledge of the distribution of X n:n determines F completely. This is true 
since F n:n (x) = [F(x)] n for every x and consequently Fix) = [F n n ix)^ /n . It 
is not as self-evident that the distribution of any order statistic F i:n will 
determine F. The proof of the assertion involves the representation (2.2.15); 
i.e., 


= / 
■'n 


Fix') n - 

o (* — 1) !(/* — f) ! 


/'-‘(l - ty~'dt. 


If two distributions F and F’ differ at some point x 0 , then clearly the 
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corresponding order statistic distributions F i:n and F[. n will differ also at 
that same point x 0 , since the integrand in (2.2.15) is positive. Knowledge of a 
few moments of F i:n will not be adequate to determine F, although it may 
allow construction of useful bounds on F. Such a program involving the mean 
and variance of F n . n and the Tschebyscheff inequality is discussed in Exer- 
cise 1. Knowledge of all moments of F i:n may under certain circumstances 
completely determine F i:n and hence F. Rather than assume knowledge of 
all moments of one order statistic, we might hope to characterize F by 
knowledge of a few moments of many order statistics. We expect to actually 
need a countable number of conditions, since F needs to be determined at a 
countable number of points. The best known result of this type is one due to 
Hoeffding (1953). It, together with a variety of variations, is the focus of our 
next section. 

Before turning to that result, it is interesting to speculate whether the 
distribution of a single spacing (instead of a single order statistic) might be 
enough to determine the parent distribution. In this case the answer is no. 
An example was provided by Rossberg (1972). For this example consider the 
spacing X 2 . 2 - X l:2 based on a sample of size 2. If the common distribution 
of the X t ’s is standard exponential then X 2:2 - X i:2 has a standard expo- 
nential distribution. Unfortunately, there are different parent distributions 
(not exponential) for which the spacing X 2 . 2 - X 1 . 2 also has such a standard 
exponential distribution (see Exercise 12). 


6.3. CHARACTERIZATIONS BASED ON MOMENTS 
OF ORDER STATISTICS 

Suppose X v . . . , X n are i.i.d. F with finite mean /x. The representation 
(5.2.8) of E{X l n ) is most convenient for our present purposes; i.e., 

M, :n = fF- l (u)g i:n (u) du, (6.3.1) 

'o 

where g i n (u ) is a BetaO', n + 1 - /) density. The question at issue is: To 
what extent does complete or partial knowledge of the array {/x ( n : i = 
1, 2 n = 1,2,...) determine F~ 1 and hence F? 

Hoeffding’s original proof that the entire array does determine F pro- 
ceeds as follows. For each n, consider a discrete distribution which puts mass 
1/n at each of the points Mi :n ,M 2: n, ■ • • > M n;n - Call this distribution F*. 
Now show that F* converges weakly (in distribution) to F, and thus F is 
determined. This is a nontrivial result and requires care in verification. The 
following heuristic argument lends plausibility to the result. First note that, 
for large n, E(X r n ) = F~ l (r/n ) [this is plausible since if « -» « and r -» °o 
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so that Vn (r/n - p) -» 0 then Jn[X r . n - F '(p)] N(0, p( 1 - p))]. It fol- 

lows that for any x with F* defined previously, 


1 " 


F n *(x) = -Ll(E(Xr:n)Zx) 


i- 1 




but this last expression is simply an approximating sum to a Riemann integral 
on (0, 1). Thus 


F*(x) = J 1 I(F '(u) £x)du 

= f l /(u <, F(x)) du 
J o 

= F(x). 


The second approach to verifying that {p,- . „} determines F 1 involves the 
concept of a complete family of polynomials in (0, 1). It is well known that if 



k = 0, 1,2 


(6.3.2) 


then gix) = 0 a.e. on (0, 1) and, if g is right continuous, it will be thus zero 
everywhere on (0, 1). This result is often described as an indication that the 
class of polynomials (1, x, x 2 , . . .) is complete in the space of integrable 
functions on (0, 1). Of course, other families of polynomials are complete, 
and, in fact, even certain subsequences of [x n ] are complete. A famous result 
of Muntz says that U"‘} is complete iff E*_ 1 = «. Let us review how these 
completeness results allow us to determine F~ l from a knowledge of the 
moments of the order statistics {p i:n ). First note that there is a great deal of 
redundant information in the full array {p, . For example, the triangle rule 
(5.3.13) with m — 1 becomes 

(« - 0 Pi-.n + '>i + l:n = (6.3.3) 

Thus, within the little triangle formed by the three order statistics moments 
appearing in (6.3.3), knowledge of any two determines the third. It thus 
follows readily that the full array {p, .„} will be completely determined if we 
know just one term in each row. It turns out to be convenient to focus on 
maxima. The result then takes the form of Theorem 6.3.1. 
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Theorem 63.1. Let F and F' be arbitrary distributions. For n = 1, 2, . . . 
denote by X n:n (respectively, X' nn ) the maxima of n i.i.d. random variables 
with common distribution F (respectively, F'). Assume E{X n :n ) = E{X' nn ), 
n = 1,2, ... (assume finite). It follows that F(x) = F'(x), x e R. 

Proof. We have / 0 1 F _1 (u)u n_1 du = / 0 1 F' _1 (u)u n_1 du for n = 1,2,...; 
ie., /otF _1 (u) - F'~\u)]u J du = 0, / = 0, 1,2, ... . But by completeness of 
u‘, it follows that F _1 (u) s F'~\u) and thus F = F'. □ 

The more general results are that 

(i) X « i determines F (i.e„ one /x,.„ from each row). 

(ii) {fXk .n x determines F where k is fixed and e?-i = 

Curiously, it is known to be possible in (i) to delete and still 
characterize F, but it is not known whether just the tail of the sequence 
/**<„):„ would suffice. Huang (1989) provides a clear survey of this and related 
problems. 

For example, if we know that = i/{n + 1) for every i and n, then 
we can conclude that F is Uniform(0, 1). We of course only really need to 
know that = l/(n + 1), n = 1,2, ... , or more esoterically that fi l:n = 
1 /(«, + 1), i = 1,2,... for some sequence n, with E” =1 n, rl = «. Other 
examples are discussed in Exercises 2 and 3. 

Rather than being given the actual moments we might be given some 
interrelations among them. Again we may ask to what extent information of 
this nature can characterize F. For example, we might be given information 
about expectations of certain spacings. It is not hard to verify using the 
triangle rule that knowledge of the expectation of just one spacing for each 
sample size n = 2,3,... is sufficient to determine F up to possible transla- 
tion (Exercise 4). It seems doubtful that knowledge of the expectation of just 
one spacing for each of a subsequence {«,} of sample sizes will pin down the 
distribution F (even assuming E” = x ny 1 = «). However, a counterexample 
does not leap to mind. 

Any set of information which will enable us to complete or essentially 
complete the triangular array {/!, .„} will, of course, suffice to determine F. 
More general expected spacings, ratios of means of order statistics, means of 
functions of order statistics, all are potential fodder for the characterization 
mill. As a final example we may ask: For which distributions F is it true that 


n — 1 

M n:n - Mi:„ = ^jTj-; n = 2,3,...? (6.3.4) 


Rephrased, is knowledge of the expected range for every sample size ade- 
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quate to determine FI Note that if Fix) satisfies (6.3.4), then so does 
Fix + c), for every c. So we may without loss of generality assume = 0 
in addition to (6.3.4). Now it is clear that (6.3.4) is satisfied by order statistics 
from a Uniform(- j,\) distribution. Unfortunately, it is also satisfied by a 
broad spectrum of distribution functions. To see this, consider the following 
question. Under what circumstances (i.e., for what function h ) will the 
inverse distribution function 


F~ l (u) = u — j + h(u) 


satisfy ju, 1:1 = 0 and (6.3.4)? 


6.4. CHARACTERIZATIONS BASED ON DISTRIBUTIONAL 
RELATIONSHIPS AMONG ORDER STATISTICS 

The classic example of this type was first explicitly stated by Desu (1971), 
although it was surely known much earlier. He observed that for samples 
from an exponential distribution, one has 

nX Un =X I:,, n - 1,2 (6.4.1) 

Obviously a distribution degenerate at 0 satisfies (6.4.1) but, except for this 
trivial case, Desu argued that (6.4.1) can only be satisfied if X is an 
exponential variable. 

The argument used is as follows. First (Exercise 5) it is clearly impossible 
to have (6.4.1) satisfied unless F( 0) ^ 0; i.e., the A'/s must be nonnegative 
random variables. Then if (6.4.1) holds for two integers m and n, we have 


[ f (^)] =F 

(6.4.2) 

and 

r / v -i/rt 

\F — | =F(x) 

L \m ) J 

(6.4.3) 

for every x. If we replace x/m by y in (6.4.3) and rearrange, we conclude 
that 

F(my) = (F(y)) m 


and rearranging (6.4.2) we have 






CHARACTERIZATIONS BASED ON DISTRIBUTIONAL RELATIONSHIPS 


147 


Combining these equations, we may conclude that for every positive rational 
number q we have 


F(q X ) = [Ft*)] 9 . (6-4.4) 

In particular, (6.4.4) holds for x = 1. It is not hard to verify that F(l) must 
satisfy 0 < F(l) < 1, to avoid degenerate solutions. Consequently, we may 
define 6 > 0 by 


F(l) = 

and conclude that our survival function satisfies 

F(q) = e~ q/e (6.4.5) 

for every positive rational q. Since the rationals are dense and since F is 
right continuous, (6.4.5) must continue to hold for every real positive q, and 
the exponential character of the distribution is established. 

A careful look at the proof suggests that we do not need to assume (6.4.1) 
holds for every n. It is enough that it hold for two integers n, and n 2 which 
are relatively prime and distinct from 1. What if (6.4.1) only is assumed to 
hold for one value of nl For example, suppose we assume 2X x . 2 = A",.,. 
What do the nonexponential solutions look like? The following construction 
is possible. Pick an arbitrary positive number in the interval (0, 1), say 8. Now 
let g be an arbitrary right-continuous nonincreasing function defined on the 
interval [1,2] with the constraint that g(l) = 8 and g(2) = 8 2 . Now extend g 
to a function G on (0, oo) by setting G(x) = g(x) on (1,2] and using the 
relation 

G(2x) = [G(*)] 2 (6.4.6) 

repeatedly^Thus, use the values of G in (1,2) and (6.4.6) to determine the 
values of G in the interval (2, 4); then use their values and (6.4.6) again to get 
the values of G in the interval (4,8), etc. Note that also (6.4.6) can be 
rewritten as 


BfD-iffwr. 

so it can be used to determine values in the interval (|, 1), then (^, |), etc. 
Formally, the function so constructed is given by 

G(x) = [g(2-'x)} V , if 2 J < x < 2 j+I , j = 0, ± 1, ± 2 (6.4.7) 
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Evidently, the function G is nonincreasing and right continuous with G(0) = 1 
and G(oo) = 0, and so is a valid survival function satisfying 2X X:2 = A",. It 
will only be exponential if the original relatively arbitrary function g was 
itself of the form g(x) = S x ( = e~ x/9 ). If g is not of the form S x , then the 
function G will behave quite erratically as x approaches 0. It will be right 
continuous, but it will not have a right derivative. If we put on the extra 
regularity condition that the survival function should have a right derivative 
at 0, then it may be verified that only exponential solutions are possible (see 
Exercise 6). 

A minor variation of the Desu theme involves an assumption that 

A\:n = a n + b„X x . A 

for every n. Distributions which satisfy this relation are known as min-stable 
distributions. (Of course, there are also analogous max-stable distributions.) 
We will meet them in our study of the asymptotic distribution of sample 
extremes (Section 8.3). The only three possible distributions for X x . , can be 
identified by referring to Theorem 8.3.5. 

After our detailed analysis of the consequences of (6.4.1), we are in a 
position to visualize the kinds of results to expect when other distributional 
relationships are assumed to hold among order statistics. Suppose instead we 
postulate that for some fixed i, j, m, n, and some fixed c (# 0, - 1, 1 to avoid 
trivialities), we have 


X,:. i cX j:m . (6.4.8) 

What can be said about the common distribution F of the A'/s? Recall that 
X i:n — Consequently, if we denote the distribution function of 

U i:n by G i:n (x) (an incomplete Beta function, although that is not of 
importance here), we can rewrite (6.4.8) in the form 

G i:n (F(x))=G y;m (F(^)). (6.4.9) 

Consequently, if (6.4.8) holds, then the common distribution F of the X/s 

must satisfy the following functional equation: 

F(x) = <*(f(^)), (6.4.10) 

where <f> = G~i °G j n is a continuous mapping from (0, 1) onto (0, 1). We 

can actually write the right side of (6.4.10) as a composition of three 
functions by introducing the trivial function S e (x) = x/c. Thus F = <f>° F°S C 
and the corresponding inverse F -1 satisfies F -1 = S~ l ° F _1 ° This can 
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be rewritten in the form 


F-\u) =cF- 1 (^(m)) ) (6.4.11) 

where i p = <f>~ 1 = G~l, °G i:n . However (6.4.11) is well known as Schroder’s 
functional equation [see Kuczma (1968)]. Typically, such equations are dis- 
cussed under smoothness regularity conditions. Before invoking such condi- 
tions, we may speculate on the possibility of constructing “ugly” solutions to 
(6.4.8) to convince ourselves that we do expect a need to invoke some 
regularity conditions. We can in fact get such ugly solutions in a manner 
entirely analogous to that used earlier to obtain (6.4.7). Pick an arbitrary 
number 5 0 in (0, 1) and set Hi) = 5 0 - Now without loss of generality assume 
c > 1. Define 5, by 


— <f>(8 0 ), 

where <f> is as defined following Eq. (6.4.10). Now set F{c) = 5, [it is not hard 
to verify that in order to have (6.4.8) hold with c > 1 we must have 
5 0 < 5, < 1]. Now let t) be an arbitrary nondecreasing right-continuous 
function satisfying tj(1) = 8 0 and rj(c) = 5,. Next define F by 

F(x) = f](x), 1 < x < c, 

and then extend F to the interval (c, c 2 ) by using (6.4.10), then to (c 2 , c 3 ), 
etc. Analogously, use (6.4.10) to extend F to (c~‘, 1), (c~ 2 , c -1 ), etc. Finally, 
set Fix) = 0, x < 0. The result is a relatively arbitrary distribution function 
with support on (0,°°) which will be a solution to (6.4.10). But it will be an 
unsatisfactory solution, since the function misbehaves near 0. If we enthusias- 
tically impose a differentiability condition on solutions to (6.4.10), then we 
are led to the following simple one-parameter family of solutions (expressed 
in terms of F~ l which will determine F ): 

F~'(u) = c 0 lim c n ifi (n) (u), (6.4.12) 

n -»oo 

where denotes the n-fold iterate of the function i p defined fol- 
lowing (6.4.11) and where c 0 is a positive real number (actually it is 
id/du)F~\u) | u _ 0 )- 

There is a fly in the ointment here. The solution (6.4.12), although easily 
verified to indeed satisfy (6.4.11), will be of limited utility unless we can 
actually get a grasp of the form of i Few functions have well-behaved 
iterates. Bilinear functions [of the form (a + bu)/(c + du)] do and so do 
functions of the form u y . There are not many others. The situation is worse 
than that, since not only do we need to have i p amenable to analytic iteration, 
but also it must be of the form Gf] n °G i n . In this sense (6.4.12) serves 



150 


CHARACTERIZATIONS USING ORDER STATISTICS 


essentially as an existence theorem. If you stumble on a distribution which 
satisfies (6.4.8), then subject to differentiability, it is unique. The cases 
X l n = cX l:m and X n . n = cX m:m [which are essentially equivalent to (6.4.1)] 
remain our prize examples. 

With some trepidation we may now explore the consequences of distribu- 
tional relations between linear combinations of order statistics. Suppose that 
we know, for certain constants a i:n , b j:m , that 

n , m 

E (6.4.13) 

f-i y=i 

What can be said about F, the common distribution of the A,’s? Note that 
we assume that the A/s in both samples have the same distribution F. First 
it should be determined that the problem at hand is not vacuous. Are there 
any interesting (nontrivial) cases in which a relation such as (6.4.13) does 
hold? There are indeed several, though there are not as many as one might 
expect. First consider samples from exponential distributions. In such a case 
spacings are again exponential and we can write 

Xi-.H-Xj-UH =c(i,y,/n,n)(A,. m - A,_ 1:m ) (6.4.14) 

for a suitable constant c(i,j, m, n), clearly an instance of (6.4.13). Is (6.4.14) 
enough to guarantee a common exponential distribution for the A,’s? An- 
other example involves samples from a uniform distribution. Again focus on 
spacings. One may verify that in such a case we have 

X i+k:n -X j:n i(X i+k:n -X i:n ), (6.4.15) 

since both sides will be scaled Beta random variables. Does (6.4.15) charac- 
terize the uniform distribution? 

Our final example involves observations from a logistic distribution Fix) = 
(1 + e~ x/a )~ l . It is true, though far from obvious [see George and Rousseau 
(1987)], that in such a setting 

X U3 + X 3:3 = 2A 2:3 . (6.4.16) 

Is this only true for logistic samples? 

Two comments should be made to focus on potential difficulties inherent 
in trying to solve (6.4.13). First we may note that (6.4.8), which we were only 
able to incompletely resolve, is a special case of (6.4.13) and, second, exact 
distributions of linear combinations of order statistics are generally not easy 
to write down. The exceptional case involves spacings and higher-order 
spacings. For these we can at least write a simple integral expression for their 
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distribution. Since two of our examples involved higher-order spacings, it is 
reasonable to first focus on such statistics. So for the moment consider the 
simpler condition 


X i+k:n -X i:n == c[X j+l m — Xj . m ] ; (6.4.17) 

i.e., two higher-order spacings have identical distributions up to scale change. 
If we assume that the common distribution F of the X/s is absolutely 
continuous with density /, then (6.4.17) implies equality of the corresponding 
densities of the random variables. Thus we have 


i!l ? Tii< ; *■) -n-)] 

X[1 -F(u +x)] n ~'~ k f(u)f(u +x)du 


k - 1 


m\ 


(;-l)!(/-l)!(m -/-;)! 
1 
c 


X-/ [F(u)] ; 

C •'-00 
X/(n)/|n + -)dn. 


( 1 

i- 1 

r / x \] 

F « + - — F(u) 


1 -F \u + - 

. \ cl . 


l c }, 


m —} — l 


(6.4.18) 


This expression is quite complicated. Only certain special cases of it have 
been resolved in the literature. The obvious modification in which we restrict 
attention to true spacings, i.e., k = 1 and / = 1, is more appealing. The 
simplified version of (6.4.18) may then be written as 


f {F{u)] i - i f(u)lF(u+x)] n ~ i ~ l f(u+x)du 

J — 00 

= c'f [F(n)r7(«) 

J — 00 

for a suitable constant c'. It is obvious that (6.4.19) can be satisfied by an 
exponential distribution, for then F(u + xX= e~ (u+x)/9 ) factors into a func- 
tion of u and a function of x. The other functions involving u + x and 
u + x/c also factor, and eventually the result is verified for suitable c and c'. 
Nonexponential solutions do exist. For example, if n = m, then uniform 
spacings for differing i and j’s will satisfy (6.4.20) (each such spacing has a 
scaled Beta distribution). If F has (0, °°) as its support, it is quite plausible 
that an assumption like (6.4.19) or even (6.4.18) should be enough to 
guarantee that F is exponential. Only a few well-behaved cases have been 


_/ x\ x\ 

F u - 1 — / u -t — I du (6.4.19) 

> c . C / 
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settled. For example, it is true that if X jn =X l . n _ j+l , then 

provided F is nondegenerate and is not a lattice distribution, it must be 
exponential [Rossberg (1972)]. His proof is nontrivial. If we are willing to 
assume a monotone hazard rate, then an easier proof may be devised 
(Exercise 7). Another result of this type states that under mild conditions, 
X j;n - Xj_ 1:n = cXj. j is enough to guarantee a common exponential distri- 
bution. Several of these results can be related to a currently active research 
area involving integrated Cauchy functional equations. Recall the Cauchy 
functional equation (on R + ) is of the form 

f(x+y)=f(x)f(y), Vx,y> 0, (6.4.20) 

and, subject to mild conditions, has as its only solutions functions of the form 
e~ Xx . Suppose that instead of requiring (6.4.20) to hold for every x and y, we 
ask that it hold on the average for every x. Thus, for some distribution G we 
ask that 


/ /(* + y ) dG(y) = j f(x)f(y) dG(y) 

J o J o 

= cf(x). (6.4.21) 

Recent articles by C. R. Rao and his colleagues have focused on character- 
izations using the integrated Cauchy equation (6.4.21). Equation (6.4.21) 
clearly has fix) = e~ x/e as a solution. Under mild conditions, it may be 
verified that only such exponential solutions exist. If we write the condition 
Xj.„ - Xj _ 1;n - X 1:n _ j+1 in terms of the common survival function F of the 
Ays, it becomes 


/>(* + «)]"■'“ <av_,:.(“) - (6.4.22) 

J o 

Thus [F(x)]"'y +1 satisfies the integrated Cauchy functional equation and 
consequently Fix) = e~ x/a for some 8. It is possible that other special cases 
of (6.4.19) can be resolved by relating them to variants of the Cauchy 
functional equation. They all (when written in terms of the survival functions 
of the spacings) reduce to expressions of the form 

/ o ”[F(x + u)\ a dG,iu) = + u^dGM), 

with c ¥= 1, and it seems unlikely that nontrivial nonexponential solutions 
could exist. 
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6.5. CHARACTERIZATIONS INVOLVING DEPENDENCY ASSUMPTIONS 

Our focus in this section is on the consequences of certain dependency 
assumptions among order statistics and functions of order statistics. Two 
peculiar properties of exponential samples may be used to set the stage for 
our discussion. For an exponential sample, any two nonoverlapping spacings 
will be independent. Is such the case only for the exponential distribution? 
Again, for an exponential sample, the regression of X j n on X i n ( j > i ) is 
linear. Can this happen in other settings? 

The simplest case in which we can study these phenomena involves a 
sample of size 2. Suppose we know X x2 and X 2:2 — X x . 2 are independent. 
What can we conclude about the common distribution of the X/s? For 
simplicity, let us focus on the case where the X/s have a continuous strictly 
increasing distribution function. In Exercise 8, it will be verified that discrete 
solutions (geometric) do exist. When F is continuous and strictly increasing 
with support (0, «), the argument is straightforward following Galambos and 
Kotz (1978, pp. 46-47). For each positive y (which is a possible value of X x x 
since F is strictly increasing), we then may write 

Pi *!.! - * 1:2 > * l *,:2 = V ) = **(* 2:2 - * 1:2 > *) ( 6 - 5 . 1 ) 

for each x > 0. The left-hand side can be written as F(x + y)/F(y) (using 
the Markov property of order statistics), and the right-hand side of (6.5.1) 
must be equal to the limit of the left side as y_-> 0, i.e., F( x). Consequently, 
F( x + y) = F(y)F(x) for every x, y > 0. So F is a right-continuous (actually 
continuous) function satisfying the Cauchy functional equation and, hence, is 
necessarily of the form e~ x/e . Since the condition “X 2:2 - X x . 2 and X x 2 
are independent” is unaffected by change of location, the final result is 
slightly more complicated but straightforward. 

Theorem 6.5.1. If F is continuous and strictly increasing on its support 
and if X 2:2 — X x 2 and X x 2 are independent, then F( x) = e~ (x ~ ll)/e , 
x > fi for some positive 8 and real fi. 

It is possible to verify that the only other possible cases in which X 2 . 2 - 
*i : 2 and X x 2 are independent are the geometric random variables with 
general support sets discussed in Exercise 8. So only exponential and geomet- 
ric solutions are possible. 

If we move to larger samples, we are led to conjecture, for example, that 
independence of X i:n and L":‘ x c J (X l+J n - X i+j _ x .„) should be enough to 
characterize the exponential distribution among continuous distributions. 
Rossberg (1972) was able to prove this result but needed a technically 
complicated argument involving characteristic functions. It is undoubtedly 
true, but probably difficult to prove, that independence of any two nontrivial 



154 


CHARACTERIZATIONS USING ORDER STATISTICS 


functions of spacings is enough to guarantee that the common continuous 
distribution of the Xj’s is an exponential distribution. 

Now we turn to possible regression characterizations. For an exponential 
sample it is true that for j > i and x > 0 

E(X j:n \X i:n =x)=c+x. (6.5.2) 

Is this true only for exponential samples? The special case i = 1, j = 2 was 
first resolved in an article by Ferguson (1967). However, even more general 
cases are quite straightforward. For simplicity, let us assume that F is strictly 
increasing on (0, °°). Then for x > 0 (6.5.2) can_be written quite simply, taking 
advantage of the representation E(Z) = JqF z (z) dz for any nonnegative 
random variable Z. First note that using the Markov property of order 
statistics 


P(X J:n - X i:n > y\X i:n =x) = F(^_ (:n _, > y) 

where the T’s correspond to observations from the truncated distribution 
F Y (y) = Fix + y)/F(x). Next use a representation analogous to (2.2.15) to 
write 


p{ Y j-,.n-i >y) = j 


_ fF(x+y)/F(x)_ 


U 


(n - i)! 

^ t n ~Hl - t) 

i-i-lV. (n-j)l V ' 


i-i-i 


dt. 


From this we obtain the following statement equivalent to (6.5.2). For every 
x > 0, 


C f F{x+y)/nx} t(t) dtdy = c, (6.5.3) 

J o J o 

where tj/it) = [(« - i)\/(j - i - l)!(n - j)\]t n ~Kl - O' - '" 1 . It is not trans- 
parent in general that (6.5.3) is enough to guarantee that F is exponential, 
but it is easy when j = i + 1. In that case ift(t) = in — i)t n ~‘~ l and (6.5.3) 
can be written as 


f“[F(x + y)/Fix)Y~‘dy = c. 

Multiplying both sides by [F(x)]" - ' and setting z = x + y in the integral 
yields 
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Differentiation of both sides of this expression (using the fundamental 
theorem of calculus), yields a differential equation whose only solutions 
correspond to exponential survival functions. 

More generally one might consider linear regressions of the form 

E{Xj, n \X i . n = x) = a + bx, (6.5.4) 

where a > 0 and b > 1. Exercise 9 focuses on such problems when j = i + 1. 
The problem is open when j > i + 1. 

There are, of course, discrete solutions to (6.5.2). They turn out to be 
geometric, so there are no surprises in that direction. The general result is 
that independence of functions of spacings or lack of correlation of functions 
of spacings is a rare phenomenon. When it is encountered, it is almost always 
associated with exponential or geometric random variables. 


6.6. CHARACTERIZATIONS INVOLVING SAMPLES OF RANDOM SIZE 

Rather than a fixed number n of observations A',, X 2 , . . . , we now consider 
cases where a random number N of X's are observed. Here N is a positive 
integer-valued random variable with 

P(N = n) = p n (6.6.1) 

and corresponding generating function 

oo 

p n(s) = E P n s n - (6.6.2) 

n = 1 

Attention will be focused on the corresponding sample extremes, in particu- 
lar on the sample minimum; i.e., 

y=minA , i , (6.6.3) 

i ' <,N 

since analogous results for the maximum are obviously obtainable. Generally 
speaking, Y will have a distribution markedly different from the common 
distribution of the A'/s. Exceptions do occur. For a given choice of distribu- 
tion for N [i.e., a given generating function P N (s)\, we may reasonably seek 
out distributions for the X’s such that 

y[ = min A’,) = aX + b. 

' izN ’ 


(6.6.4) 
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Rather than try to present a complete solution to this problem, let us 
focus on the special case where b = 0 and where we assume that the 
common support of the X/s is (0, »). Thus our scenario is as follows. N is 

assumed to be a nondegenerate variable with possible values 1,2 the 

X’s are i.i.d. positive random variables, and for some a > 0 

Y = aX. (6.6.5) 

By conditioning on N we find a relationship between the survival functions 
of X and Y. Thus, 


F Y (y) = H P(Y > y\N = n)P(N = n) 

n-1 

= L {F x (y)] n Pn 

n = 1 

= P z {F x (y))- 

Consequently, (6.6.5) is equivalent to 

F X (x/a) = P z {F x (x)). ( 6 . 6 . 6 ) 

Aha! We have seen equations like this before; cf. Eq. (6.4.10). Denote the 
inverse of F x by F x 1 and (6.6.6) can be rewritten as 

F x \u)~aF x l {P z (u)). (6.6.7) 

This, of course, is just like (6.4.11) (the Schroder equation), only now F x l is 
decreasing whereas in (6.4.11) F~ 1 was increasing. As earlier, mild regularity 
conditions yield a solution of the form 

F x l (u) = c„ lim a n P^ n) (u). (6.6.8) 

ft-* » 

The classic example in which the iterates Pz } are available in closed form 
corresponds to the case in which N has a geometric distribution, and the 
corresponding distribution for X [and also Y in (6.6.5)] is log-logistic (see 
Exercise 10 for the necessary details). 

The equation min i:S N AT, = X + b can be reduced to (6.6.5) by exponenti- 
ation; i.e., let XI = e x ‘, X' = e x , and a = e b . Thus, for example, one may 
verify that a geometric minimum of logistic random variables is again logistic 
but with a new location parameter (Exercise 10). 

The logistic distribution has another curious characteristic property associ- 
ated with samples of random size. Assume again that N is geometric and 
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define Y = min, sN A", and Z = max l sN A',. Both Y and Z will be logistic 
and, in fact, one may show that under mild regularity conditions 


( min A'. l 

d / \ 

1 = max XA + c 

'isiv > 

X i<N ’ 


only if the A'/s are logistic random variables (Exercise 11). 


EXERCISES 

1. Suppose that A',, . . . , X n are i.i.d. with common distribution function F. 
Assume that E(X n n ) = v n and vaKA' M:M ) = t„ 2 . Use the Tschebyscheff 
inequality to obtain an upper bound for PiX > v n + c), where c > 0. 

2. Suppose that for each n, EiX n . n )/EiX n _ ln _ l ) = c > 1. Determine the 
common distribution of the A'/s. 

3. Suppose that for each n, EiX n . n /X n _ ln ) = c n for some c > 1. Deter- 
mine the common distribution of the X’s. 

4. Suppose that E(X n:n -X„_ 1: „)= 1/n, n = 2,3, ... . Determine the 
common distribution of the A'/s. 

5. Suppose that nX x = X x . , for every n = 1,2 Show that P(X t < 0) 

= 0. 

6. Suppose that 2X i:2 =X t:l and that F(x), the common distribution of 
the A'/s, has a right derivative at zero. Prove that Fix) = e~ x/e for some 
6 > 0. 

7. Suppose that for some j < n we have X j:n - X j _ i:n = X i:n _ j+l . As- 
sume that F, the common distribution of the A'/s, has a monotone 
hazard rate. Prove that Fix ) = 1 - e~ x/e , x > 0, for some 0 > 0. 

8. Suppose that X u X 2 are i.i.d. nonnegative integer-valued random vari- 
ables. Assume that X l:2 and X 2:2 - X l:2 are independent. What can be 
said about the common distribution of the A'/s? Generalize to the case in 
which the common support set of the A'/s is of the form 0 < x, < x 2 < 

* 3 < • 

9. Suppose that for some j < n, EiX j+l . n \Xj. n = x) = a + bx for some 
a > 0 and b > 1. Under suitable regularity conditions determine the 
common distribution of the A'/s. 
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10. (a) Let X x ,X 2 ,... be i.i.d. positive random variables and assume that 

N, independent of the A'/s, has a geometric distribution. Define 
Y = min^jv-Y, and suppose that Y = aX x for some a > 0. Verify 
that under mild regularity conditions the X’s have a common log- 
logistic distribution. 

(b) With the same setup as in part (a) except that we no longer assume 
that the X/s are positive random variables, discuss the implications 
of Y = a + X y 

11. Suppose X x , X 2 , ... are i.i.d. random variables and that N, independent 
of X/s, has a geometric distribution. Discuss the implications of 


f min A r , 

1 = (max A",) 

'/«:jv ‘ 

1 iZN ’ 


12. Let X x , X 2 be independent identically distributed random variables with 
common distribution function 

F( x) = 1 - e~ x [\ + 7t _2 (1 - cos27tx)], x > 0. 

Verify that X 2 . :2 - X 1;2 has a standard exponential distribution. 

(Rossberg, 1972) 

13. Suppose X u X 2 are i.i.d. absolutely continuous positive random vari- 
ables. Identify the nature of their common distribution under the as- 
sumption that X l:2 /X 2:2 and X 2:2 are independent. Describe an analo- 
gous characterization involving n independent X/s. 

(Ahsanullah and Kabir, 1974) 

14. Suppose X V X 2 , X 3 are i.i.d. positive absolutely continuous random 
variables. Assume that X 2;3 - X I:3 = X I:2 and X 3:3 - X l;3 = X 2:2 . 
Prove that A'/s are exponentially distributed. 

(Ahsanullah, 1975) 

15. Suppose that / is a right continuous function defined on R + which 
satisfies the Cauchy functional equation (6.4.20). Prove that fix ) = c x 
for some c e R + . 

16. Suppose that /, g, h are real-valued right continuous functions defined 
on R + satisfying fix + y) = gix)hiy), Vx, y > 0. Determine the nature 
of the functions /, g, and h. 


(Pexider, 1903) 



CHAPTER 7 


Order Statistics in Statistical 
Inference 


7.1. INTRODUCTION 

Order statistics play an important role in several optimal inference proce- 
dures. In quite a few instances the order statistics become sufficient statistics 
and, thus, provide minimum variance unbiased estimators (MVUEs) of and 
most powerful test procedures for the unknown parameters. The vector of 
order statistics is maximal invariant under the permutation group of transfor- 
mations. Order statistics appear in a natural way in inference procedures 
when the sample is censored. They also provide some quick and simple 
estimators which are quite often highly efficient. We now explore some basic 
facts about the use of order statistics in statistical inference (in estimation, 
prediction, and testing of hypotheses). We carry out the discussion assuming 
that we are sampling from an absolutely continuous population. However, 
some of our results hold even when the population is discrete, and this will 
be pointed out at appropriate places. 

We begin with a discussion of various types of censored samples in Section 

7.2. These include Type I and Type II censoring and some modifications to 
them. In Section 7.3 we look at the role of order statistics as sufficient 
statistics, especially when the range depends on the unknown parameter 8. 
Optimal classical inference procedures would then depend on order statis- 
tics. We also look at the amount of Fisher information contained in a single 
order statistic or a collection of order statistics when the sample comes from 
an absolutely continuous distribution. We take up in Section 7.4 the question 
of maximum-likelihood estimation of 6 when the data is a censored sample. 
Through examples, we look at the finite sample as well as asymptotic 
properties of 8, the maximum-likelihood estimator (MLE) of 8. We discuss 
the problem of estimating the location and scale parameters using linear 
functions of order statistics (known as L statistics) in Section 7.5. Following 
the work of Lloyd (1952), we provide best linear unbiased estimators (BLUE) 
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of these parameters. In Section 7.6 we discuss situations where prediction of 
order statistics is important and present the best linear unbiased predictor 
(BLUP) of an order statistic. Section 7.7 describes how order statistics can be 
used to provide distribution-free confidence intervals for population quan- 
tiles and tolerance intervals. A brief discussion of the role of order statistics 
in goodness-of-fit tests is provided in Section 7.8. In the last section we look 
at some important uses of order statistics in monitoring data for possible 
outliers, and in forming robust estimators which are less susceptible to model 
violations. 


7.2. TYPES OF ORDER STATISTICS DATA 

Let us consider a life-testing experiment where n items are kept under 
observation until failure. These items could be some systems, components, or 
computer chips in reliability study experiments, or they could be patients put 
under certain drug or clinical conditions. Suppose the life lengths of these n 
items are i.i.d. random variables with a common absolutely continuous cdf 
F{y\ 0) and pdf f(y; 0), where 0 is the unknown parameter. Then we have a 

random sample Y, Y„ from the cdf F(y\ 0). Note, however, that these 

values are recorded in increasing order of magnitude; that is, the data appear 
as the vector of order statistics in a natural way. For some reason or other, 
suppose that we have to terminate the experiment before all items have 
failed. We would then have a censored sample in which order statistics play 
an important role. Let us now look at some prominent classifications of 
censored samples discussed in the literature. 


IVpe I (Time) Censoring 

Suppose it is decided to terminate the experiment at a predetermined time t, 
so that only the failure times of the items that failed prior to this time are 
recorded. The data so obtained constitute a Type I censored sample. It 
corresponds to right censoring in which large observations are missing. 
Clearly, the number of observed order statistics R is a random variable; it 
could even be 0. The likelihood function can be written as 

L (8\r,y) = ^' r y f(yu d ) • ' /(mW 

yi < yi < ■ • ■ < y r < <> o <r<.n 
= {1 -F(r,0)) n , r = 0, t < y,, (7.2.1) 


where y is the vector of observed order statistics. 
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Having two distinct forms for L(8) complicates the study of the finite- 
sample properties of the MLE of 8, as we shall show through an example in 
Section 7.4. 


Type II (Failure) Censoring 

If the experiment is terminated at the rth failure, that is, at time Y r:n , we 
obtain Type II censored sample. Here r is fixed, while Y r:n , the duration of 
the experiment, is random. The likelihood function is 

L(0|y) = 77T=T)T /(y,; e) ‘ ‘ ‘ /( e) {1 ” F{ yr ’ 

y, <y 2 < <y r • (7.2.2) 

As in the Type I censored case, the just-mentioned likelihood corresponds 
to a right-censored sample; that is, large values are censored. One can have 
left censoring wherein smaller values are censored or one can have censoring 
of multiple regions. When there is just left and right censoring, the name 
used often is double censoring. More general censoring schemes have been 
discussed in the literature. 


Random Censoring 

With the ith item let us now associate a random variable C, called the 
censoring time whose cdf F c is free of 8. Define T i = min(Y), C,) and D i = 1 
if Tj = Yj and D i = 0 otherwise. Let us assume Y i and C i are independent, 
and we observe (7), £>,), i = 1 to n. Thus, each lifetime is censored by an 
independent time, and we also know whether our observation is the life 
length or the corresponding censoring time. This scheme is known as a 
random censoring scheme and is very common in clinical trials. In such 
experiments, patients enter into the study at random time points, while the 
experiment itself is terminated at a prespecified time. The likelihood function 
in this case can be written as 

= n{/(y,;0)[i-/ r c(y i )]}Vc(^)[i-f(^;e)]} 1_< ' , > (7.2.3) 

i = 1 

where f c is the pdf of C,. 

If the C,’s are constants, say t, then (7.2.3) reduces to 

L( 0 \t,d) = fl/(y,.;0){l -F(t;8)} n - r , 

1 = 1 



162 


ORDER STATISTICS IN STATISTICAL INFERENCE 


y, y r < t; d, , = 0, 1 and Ed, = r. In other words, we obtain a Type I 

censored sample. 

Progressive Censoring 

Of the n items put on test, suppose we remove r, unfailed items at time 

n 2 unfailed items at time t 2 where t x <t 2 < ••• are prespecified 

times. The experiment will be terminated at time /, if r, exceeds the number 
of unfailed items remaining at that time. This is known as a Type I progressive 
censoring scheme. A Type II version of this is also discussed in the literature. 
In that scheme at the time of r f th failure, r, unfailed items are removed from 
the study (i ^ 1). This is continued until each item is taken care of either due 
to its failure or due to its removal from the experiment. Obviously the 
likelihoods are more complex; but the Type II case is tractable. Lawless 
(1982, pp. 33-34) provides for some details about such schemes. 

We can discuss censored samples from discrete distributions also; but, as 
we discovered in Chapter 3, the likelihood would be more complicated than 
the ones displayed above. 


73. ORDER STATISTICS AND SUFFICIENCY 


The concept of sufficiency is fundamental in classical parametric inference 
procedures. Sufficient statistics lead us to MVUEs, optimal tests, and confi- 
dence intervals. Order statistics, being sufficient, play an important role in a 
variety of such procedures. We now discuss the role of order statistics in data 
reduction. 

Let us begin with the vector of all order statistics Y = (y,.„ Y n:n ) 

from a random sample of size n from the cdf F{y \ 0). The parameter 0 may 
be real or vector valued and belongs to H, the parameter space. When F is 

absolutely continuous, the conditional joint distribution of Y,,Y 2 Y n 

given Y l:n =y l Y n:n = y„ is easily seen to be 


^(Yl = y h Yn = VijYl-.n - V\ Y n:n=y n ) = - J, 


where (i v i 2 /„) is a permutation of (1,2 r) and y, < • •• <y„. 

When F is discrete, and Y Vn Y n:n have ties such that there are only k 

distinct y/s with frequencies r, n k , one would have 


P{Y t - y„ Y„ — y i jY l . n Y n . n y„) 


r,!r 2 ! 


Hence, in either case, the conditional distribution of the sample is free of 0. 
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Thus, Y is sufficient for 0. For some distributions Y is in fact minimal 
sufficient. For example, this is the case when F is a Cauchy cdf with location 
parameter 0. Other examples include the logistic and Laplace distributions 
(Lehmann, 1983, p. 43). 


The Role of Sample Extremes 

A subset of Y, notably either or both of the sample extremes, may become 
sufficient statistics when the range of the distribution depends on 0. Such a 
situation is referred to as a nonregular case in the literature. Early investiga- 
tion regarding the existence of a single sufficient statistic for a single 
parameter 0 in the nonregular case was undertaken by Pitman (1936). His 
work was followed up by Davis (1951) and Huzurbazar. The latter’s work is 
collected in Huzurbazar (1976). We now follow that presentation to get a 
general idea about sufficiency and minimum variance unbiased estimation in 
the nonregular case. 

Let F(y; 0) be absolutely continuous with range (aid), bid)), where 0 is a 
real-valued parameter. Let us assume both aid) and bid) are differentiable, 
monotonic in opposite directions, and aid) < bid) for all 0 e ft. Then, a 
necessary and sufficient condition for the existence of a single sufficient 
statistic for 0 is that 

f(v,0) =g(d)h(y), a(d) <y < b(d), 0 e ft. (7.3.1) 

When aid) decreases and bid) increases, the sufficient statistic is 

T=max(a- 1 (Y 1: J,b- 1 (Y n:n )). (7.3.2) 

If aid) increases and bid) decreases, it will be 

T = m in{a-'(Y l:n ),b-'(Y n:n )). (7.3.3) 

A single sufficient statistic does not exist if both the limits are monotonic in 
the same direction. When aid) is a constant, Y n:n or any one-to-one function 
of it would be sufficient. 

We now obtain the cdf of T given by (7.3.2). Consider 
F T (t)=P(T<t) 

= P( a ~ l ( Y un)^t,b- l (Y n:n )<t) 

= P(Y 1:n >a(t), Y n:n zb(t)) 

= {F(b(t);0) -F(a(t);d)) n 

= ^g(8) f^h(y) dy^ 
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on using (7.3.1). Hence, 


FA‘) = U(«)M0}". (7.3.4) 

since f$Jgi0)hiy) dy = 1 for all 0. The condition aid) <,Y l n ^Y n . n <, bid) 
yields T <, 0. Further, since ait) £ bit) whenever F T it) > 0, T ^ t 0 where t 0 
is the solution of the equation ait 0 ) = bit 0 ). 

Example 7.3.1 (the uniform distribution). Let Y be Uniform(0, 0), 0 > 0. 
Here ai0) is a constant. The sufficient statistic T = Y n n and is known to be 
complete. [See, for example, Hogg and Craig (1978, pp. 354) for a definition 
of completeness.] Hence, the unique function of T which is unbiased for 0 
is its MVUE. Now F T it) = it/0) n , EiT) = n0/in + 1), and, hence, 0 n - 
in + 1) T/n is the MVUE of 0. It is also known that a uniformly most 
powerful test exists for testing H 0 : 0 <. 0 O against the alternative H t : 0 > 0 O . 
Since the joint pdf of Y has a monotone likelihood ratio in Y n:n , the optimal 
test is based on the sample maximum and rejects H 0 if it is too large. 

Example 7.3.2. Let Y be Uniform( - 0, 0), 0 > 0. Here ai0) is -0, 
which is decreasing while bi0) = 0 is increasing. Further gi0) = (20)“ 1 and 
hiy) = 1. The sufficient statistic, T, given by (7.3.2), is max(-A r 1:n , X n:n ). Its 
cdf, obtained from (7.3.4), is F T it) = it/0) n , 0 <, t < 0. In other words, T 
behaves like the sample maximum from a random sample of size n from 
Uniform(0, 0). Hence, one can use Example 7.3.1 to obtain optimal statistical 
procedures that are based on T. 

Example 7.3.3. When Y is Uniform(0, 0 + 1), ai0) and bi0) are both 
monotonically increasing. Hence, a single sufficient statistic does not exist. In 
this case (Y,.„, Y n:n ) is minimal sufficient. 

The sufficient statistic T, given in (7.3.2), can be shown to be complete. If 
gi0) is differentiable, the MVUE of 0 is given by 0 n = T - giT){ng'iT)}~ 1 
(Davis, 1951; see Exercise 3). In Example 7.3.2, one would get 0 n = in + 
1 )T/n, a fact we observed via Example 7.3.1. 

The cdf of T derived in (7.3.4) comes in handy in obtaining confidence 
intervals for 0. Note that {gi0)/giT)) n is Uniform(0, 1) and, hence, can be 
used as a pivotal quantity for this purpose. 


Fisher Information Measure 

Let Y be an absolutely continuous (discrete) random variable whose pdf 
(pmf) fiy; 0) contains the real-valued parameter 0. Under certain regularity 
conditions, the Fisher information contained in the random variable Y about 
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^y(0) — E 


5 log f(Y-e) 
38 


I d 2 log f(Y;8) \ 

l J 
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(7.3.5) 


These regularity conditions include the assumption that S, the support of 
/(y;0), does not depend on 8 and that differentiation with respect to 8 and 
integration with respect to Y are interchangeable. 

The Fisher information appears in the Cramer-Rao inequality (informa- 
tion inequality), which provides a lower bound for the variance of an 
unbiased estimator. It also appears in the expression for the asymptotic 
variance of the MLE of 8 based on a random sample from f(y;8). The 
sufficient statistic contains all the Fisher information in the sample. 

A natural question is whether we can determine the amount of informa- 
tion contained in a single order statistic or a collection of order statistics 
from a random sample. While the recipe for deriving / Y (0) is simple, the 
details are messy in most cases. When Y is the vector of all order statistics 
from a random sample of size n, on noting that 


f 32 log f(Y i:n -,8) » a 2 log f(Yr,8) 

h M 2 h M 2 

it follows that / Y (0) = nI Y (0) where / Y (0) is the information in a single 
observation. This is true even when F(y\8) is a discrete cdf. But when Y 
consists of only a subset of the entire vector of order statistics, determination 
of the information is harder. Let us look at two examples. 

Example 7.3.4 (the exponential distribution). Let Y be Exp(0); i.e., 
/(y; 0) = 8~ 1 e~ y/e , y > 0. Then, / Y (0) = 1/0 2 . Let us compute the informa- 
tion contained in a single order statistic, Y r:n . Since 


/r:„(y;0) 

32 [ o8/ r:w (y;0) 

38 2 


n\ 


(\-e~ y/e ) r l e -< n ~ '+'»/<>' 


( r - l)!(/j - /•)! 0 


1 

T 2 


2 y 2(r - 1) y 

-n(n-r+ 1 )£ + 


0 


0 


e i 


8 (1 


-y/e 

o-y/O) 


('-l) m 2 e~ y/fl 

0 2 10/(1 -e~ y / e ) 2 


(7.3.6) 
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on simplification. Hence for r ^ 3, on recalling (7.3.5), we can express the 
Fisher information contained in Y r . „ as 


1 2(n - r + 1) 2(n - r + 1) 

hj9) = ~j 2 + -5— ~E(X, :n ) - 2 E(X r _ l:n ) 

n(n - r + 1) 

+ (f - 2)g 2 < 7 ' 3 ' 7 > 

where X i:n is the ith order statistic from a random sample from a standard 
exponential distribution. On using the expression for /n (;n in (4.6.6), (7.3.7) 
simplifies to 


I Y (0) = , , , 

T r: „V ) qZ qZ 


1 1 n(n-r+l) 

+ — = (M 2 r- 


r - 2 


2: n - 1 + <T r- 2 




r ^ 3. 
(7.3.8) 


Now recall, respectively, from (4.6.6) and (4.6.7), that p r _ 2 „-\ = 
E/^i 2 (« - 0" 1 and <r?_ 2 :n~i = “ »)~ 2 . 

When r = 1, since Y l .„ is exponential, I Y (0) = 1 /0 2 . For r = 2, on 
using (7.3.6) and taking the expectation, we obtain 


J Y 2 „( e ) ~ 02 + 


1 



2n(n - 1) 

V 2 

2 n(n - 1) 

7 2 

( n - 1 ) 
nd 2 ’ 


f ! 

7=0 (« +» 3 


/. 


1 


x=0 ( tl + jc) 


dx 


(7.3.9) 


for large n. Also, for r > 3, with r/n = p, 0 < p < 1, we can approximate 
(7.3.8) as 


2 n(l — p) , 

“ ^2 + -P)} 2 - (7-3.10) 

First, let us note from (7.3.8) and (7.3.9) that the information contained in 
a single order statistic exceeds the information contained in a single observa- 
tion, except when the order statistic is the sample minimum. The right-hand- 
side expression in (7.3.10) increases as p increases up to p 0 , and then 
decreases where p 0 is the unique solution of 2 p = -log(l - p). The value of 
p 0 is roughly 0.8. Hence, we can conclude that / y j(0) increases for a while 
and then decreases; it peaks around the 80th sample percentile, which 
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Table 7.3.1. Fisher Information in Order Statistics From N(0, 1) distribution 
for Sample Size 10 


r 

1 

2 

3 

4 

5 

6 

7 

8 

/v (0) 

mm 

4.763 

5.755 

6.345 

6.622 

6.622 

6.345 

5.755 

/ Y (0) 

m 

4.970 

6.260 

7.238 

8.005 

8.617 


9.495 


contains about (2 + 0.65n) times the information contained in a single 
observation. 

While computations are involved for I Y (0), the information contained in 
the data vector Y r = (Y 1:n , . ..,Y r:n ) is easy to find. For this, first note that 
T = + (n — r)Y r n is the sufficient statistic. Since T can also be 

expressed as E- =1 (n - i + 1X^ ; „ - we conclude from Theorem 

4.6.1 that it has a TCr, 0) distribution. On using (7.3.5), we can conclude that 
/ r (0) = r/6 2 . Since the information contained in the sufficient statistic 
matches that of the data, it follows that / Y (0) = r/6 2 . Note that the uncen- 
sored case was handled very easily earlier. 

Example 7.3.5 (the normal distribution). Let Y be N(6, 1), where the 
mean 6 is the unknown parameter. Then I Y (0), the information contained in 
a single observation, is 1. In the case of a single order statistic Y r n , I Y (0) 
can be expressed in terms of expectations of some functions of the standard 
normal pdf and cdf (see Nagaraja, 1983). Numerical integration would 
be necessary to evaluate it. Table 7.3.1 gives the values of I Y (0) and 
/ Y (0) where Y r = (Y 1:n , . . . , Y r .„) for n = 10. The latter was computed by 
Mehrotra, Johnson, and Bhattacharyya (1979). Note that I Y (6) = I Y (0) 
by the symmetry of the normal pdf, and the information contained in" a single 
order statistic always exceeds the information contained in a single observa- 
tion. Further, the first half of the sample contains more than 80 percent of 
the information. 


7.4. MAXIMUM-LIKELIHOOD ESTIMATION 

Functions of order statistics appear as maximum likelihood estimators (MLEs) 
of the parameter 0 either when S, the range of the parent distribution, 
depends on 0 or when the sample is censored. Instead of having a general 
discussion on the finite sample and asymptotic properties of 0, the MLE of 0, 
we look at several examples to highlight various aspects of MLEs based on 
order statistics. 

When the sample is censored, the likelihood equation is dlog L/86 = 0, 
where L is given by (7.2.1) for the Type I censored sample, and by (7.2.2) for 
the Type II censored sample. It does not yield closed-form solutions for most 
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distributions. In these cases the MLE has to be obtained through numerical 
methods, which can be very computer intensive. In both these cases, 8 is 
consistent for 8, and 8, appropriately normalized, is asymptotically normal 
under certain regularity conditions. In the Type I censored case, a simple 
proof of these facts may be found in Borgan (1984), and for the latter case, 
Halperin (1952), and more recently Bhattacharyya (1985) provide a proof of 
the asymptotic properties of 8. To summarize, 8 8 and (8 - 8) is asymp- 

totically MO, 1 /I(8)), where 1(8) is the Fisher information contained in the 
likelihood given by 1(8) = -E(d 2 log L/d8 2 ). Some detailed discussion of 
the MLEs for several common distributions may be found in Lawless (1982). 

Example 7.4.1 (the uniform distribution). Let Y be Uniform(0, 8). With 
a complete sample, the MLE is 0 = Y„.„. Thus, 8 is a complete sufficient 
statistic, and (n + 1)8 /n is the MVUE of 8. Since the range of the distribu- 
tion depends on 8, the regularity conditions which are sufficient to ensure the 
normality of 8 do not hold. Let us see what happens to 8 in this example. 
Consider 


P 


Y -8 

* n : n v 

8/n 



0, y < -n, 

(l + ^ ) , -n <, y ^ 0, 

1 , y>o. 


Thus, P{n(8 - 8)/ 8 <, y) -♦ e y , y <, 0. This limiting cdf is that of -X where 
X is standard exponential. As we see later in Section 8.3, this is one of the 
three possible limit distributions for the sample maximum. The normal 
distribution is not one of them. 

With a Type II censored sample where only the first r order statistics are 
observed, 8 = nY r ,Jr. If r = [np], 0 <p < 1, 8 is asymptotically normal 
(Exercise 7). 

Example 7.4.2. Let f(y\ 8) = 1, 8 < y ^ 8 + 1. Here 8 is not unique, 
since it can be any point in (Y n .„ - 1, Y 1; „). Thus, the MLE is of the form 
8 = c(Y„.„ - 1) + (1 - c)Y 1;n , 0 < c < 1 and, hence, its asymptotic distribu- 
tion depends on the choice of c. However, 8 8. When c = 0.5, 8 becomes 

unbiased, and it can be shown to be the BLUE of 8. (See Exercise 12.) Refer 
to Sarhan and Greenberg (1959) for more details. 

Example 7.4.3 (Type II censored sample from the exponential distribu- 
tion). Suppose we observe Y l:n Y r n (l < r < n) from the pdf f(y;8) 
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= l/de y/a , y > 0. The likelihood function is [recall (7.2.2)] 

t(%> ' (7=7)7 F exp { + <" - Ov,)/*}. 

0 < y, < • <y r , 

where y is the observed vector of order statistics. On differentiating the log 
likelihood and equating the derivative to zero, and using the random vari- 
ables in the solution instead of their observed values, we obtain 

. L' =l Y i:n + (n-r)Y r:n - i + l)(Y i:n - _ _ 

— (7.4.1) 

r r 

as the MLE of 0. (How do you know that the solution really corresponds to 
the maximum value of the likelihood?) From (7.4.1) and Theorem 4.6.1 it is 
clear that rd/0 has T(r, 1) distribution. This fact can be used to carry out 
tests or construct confidence intervals for 0. It also means that 0 is an 
unbiased estimator of 0 and y7(0 - 0)/0 MO, 1) as r -» ». Note that 0 is 
a complete sufficient statistic for 0. Thus 0 is the MVUE of 0, and since it is 
a linear function of the Y l:n ’ s, it is also the BLUE of 0. In Example 7.3.4 we 
have seen that 7^(0), the Fisher information contained in 0, is r/0 2 . It is a 
linear function of r and is free of n. 

The quantity r§ = + (n - r)Y r:n represents the total time on 

test when the experiment was stopped at the rth failure. This statistic plays 
an important role in the reliability literature. 

Example 7.4.4 (Type I censored sample from the exponential distribution). 
From (7.2.1), we have 


r > 0, 0 < y, < ••• <y r < t, 
\e~ n,/9 , r = 0, 

where t is the censoring time. If r = 0, the likelihood is monotonically 
increasing and, hence, 0 does not exist. But the probability of such an event, 
e~ nt/e , decreases with n. Conditioned on the event that R > 0, the MLE is 
given by 


n\ 


1 


L(6) = 


( n - r)! 0 


7 exp 


. L^Y i:n + (n - R)t 

u 


R 
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The exact distribution of 8 is complicated since R is random. See 
Bartholomew (1963) if you are curious about its pdf. It is, however, asymptot- 
ically normal. Another alternative to using 8 is to use the distribution of R 
(binomial) to obtain an MLE of 8. Since R is Bin(n, 1 - e~‘ /e ), the MLE 
based on R would be 8* = -t/ log(l - R/n). Bartholomew (1963) notes 
that inferences based on 8* are highly efficient when compared to those 
based on 8 whenever e~ ,/e is not close to zero. As in the Type II censored 
case, r§ represents the total time on test and 8 is asymptotically normal. 

Example 7.4.5 (the normal distribution). Suppose we have a Type I 
right-censored sample from a N(8, 1) population. Assume that it consists of 
the first r (> 0) order statistics which are less than t, the censoring point. 
Then from (7.2.1) we have 

L (0) - ^- r) 7{n»(y. - ^jl 1 - *(' - *)]"“'• 

where <p and <I> are the standard normal pdf and cdf, respectively. The 
likelihood equation d log L/38 = 0 reduces to 


<p(i ~ 0) 

1 -<t>(t-8) 


r 


i=i 


(7.4.2) 


This means the ML estimate has to be determined by an iterative process. 
Cohen (1959, 1961) has facilitated this through a graph and a table, which he 
developed for the maximum likelihood estimation of the mean and the 
variance when both are unknown. Details about his procedure may be found 
in Balakrishnan and Cohen (1991, pp. 146-159). 

When we have a Type II censored sample, the ML estimate satisfies (7.4.2) 
with y r:n replacing t. That is. 


<p{y r :n ~ 0) 

1 



(7.4.3) 


Here again, Cohen’s work comes in handy. Since the failure rate function 
which appears on the left side of (7.4.3) is messy, suggestions have been made 
to replace it by its mean [Mehrotra and Nanda (1974)] or by a linear function 
of Y r:n - 8 [Tiku (1967)]. The resulting equation yields estimates of 8 known 
as modified ML estimates in the literature. For more details about such 
estimators one may refer to Tiku, Tan, and Balakrishnan (1986). 
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7.5. LINEAR ESTIMATION OF LOCATION AND SCALE PARAMETERS 

Linear functions of order statistics, known as L statistics, provide highly 
efficient estimators of location and scale parameters. The general theory of 
best linear unbiased estimation developed for linear models applies in a 
natural way to produce estimators of these parameters which are the best 
among unbiased L statistics. In this section we derive formulas for these 
BLUEs based on the finite-sample properties of order statistics. We also look 
at some examples. The asymptotic properties of L statistics are discussed in 
Section 8.6. 


Best Linear Unbiased Estimators 

Let y„y 2 ,...,y„ be a random sample from the absolutely continuous cdf 
F(y;0,,0 2 ), where 0, is the location parameter and 0 2 > 0 is the scale 
parameter. Let Y denote the vector of the order statistics in the sample. We 
will now obtain estimators of 0, and 0 2 which are the best among the 
unbiased linear functions of the components of Y. The procedure is based on 
the least-squares theory originally developed by Aitken (1935). Lloyd (1952) 
first applied the general results in the order statistics setting. 

Let A' = (Y-0,)/0 2 be the standardized population random variable 
with cdf F 0 (x) = F(x;0, 1). Clearly, F 0 is free of the parameters and, hence, 
the means and the covariances of the order statistics from the X population, 
M, :n and are free of them as well. Let X denote the vector of Z-order 

statistics corresponding to Y. Then, it is clear that 

E(Y>:n) =0. + 0 2 M,:„ (7.5.1) 

and 

cov(y i: „,^,„) = 0 2 2 a- ( , y: „, (7.5.2) 


for 1 < i, j < n. 

Let p. be the mean vector of X, and 8' = (0,, 0 2 ) be the vector of the 
unknown parameters. Further, let 1 be an n X 1 vector whose components 
are all l’s. Then the n equations in (7.5.1) can be expressed in the matrix 
form as 


E( Y) = A8, (7.5.3) 

where the n X 2 matrix A = (1, ji) is completely specified. Also, (7.5.2) can 
be put in the form 


cov(Y) = 0|S. 


(7.5.4) 
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where cov(Y) represents the covariance matrix of Y, and £ = (<r ( ; .„), the 
covariance matrix of X, is known. 

Suppose the goal is to choose 0, and 0 2 so that we minimize the quadratic 
form 


0(6) = (Y - A0)'£ -1 (Y - A6) 

= (Y- 0,1 -0 2l i)'£-'(Y- 0,l-0 2( i) 

= Y'£ -1 Y - 20,1'£ _1 Y - 20 2 (i'£"‘Y + 20 1 0 zt i'£“ 1 l 

+ 0fl'£-‘l + 0l|if£- | |fc. (7.5.5) 

If 0(8) is minimized when 0 = 6 = (0„ 0 z y, we say that 0, and 0 2 are the 
best linear unbiased estimators (BLUEs) of 0! and 0 2 , respectively. 

On differentiating (7.5.5) with respect to 0, and 0 2 and equating to 0, we 
obtain the normal equations as 

(l'£-'l)0, + (|i'£-'l)0 2 = l'£ _1 Y, 

(p.'S-'l)©, + (|fc'£-V)0 2 = p.'£ _1 Y. 

On solving these equations for 0, and 0 2 , we obtain the solution 

0, = -pTY and 0 2 = 1TY, (7.5.6) 

where T = £ _1 (lp,' - p,l')£ _1 /A and A = (l'£ -I lX|i'£ - V).- (l'£ _1 |i) 2 . 
Note that T is a skew symmetric matrix. Further, (7.5.6) can be expressed as 
the matrix equation 0 = (A'£ _1 A) _1 A'£ _1 Y where 0 = (0„ 0 2 y. Now, we will 
show that 0(0), given by (7.5.5), is actually minimized when 0 = 0. For this, 
let us write 

0(0) = (Y-A6 + A(0 - 0))'£-‘(Y- A0 + A(0- 6)) 

= 0(6) + 2(Y - A0) £ _1 A(0 - 0) + (0 - 0)'A'£-‘A(0 - 0). 

(7.5.7) 

Since Y'£ _1 A - 0A'£ _1 A = 0, the middle term in (7.5.7) vanishes. The last 
term there is always nonnegative, since £ is positive definite. Hence, we can 
conclude that 0(6) attains its minimum when 0 = 6, which means 0 is in fact 
the BLUE of 0. 

Using (7.5.4) and (7.5.6), it is easily seen that 

var(0,) = 0||fcT£r|fc = 0 2 ji'£~V/A, 
var(0 2 ) = 0|lT£n = 0ir£-‘l/A, 


(7.5.8) 

(7.5.9) 
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and 

cov(0 lt 0 2 ) = -0fpTXn = -flftfS-VA, (7.5.10) 
on simplification. These results can also be obtained by noting that cov(8) = 


When the pdf of the standardized random variable X is symmetric around 
the origin, further simplification is possible. In that case, (X l: „,..., 
X i: X nn ) = (~X n: —X„_ i+t: „, -X l:n ), which can be repre- 


i . n ' ' n . n 

sented as 

' n . n ' ' n — i t i . n ' 

X = -JX 


r . ft ' * 

(7.5.11) 

where 

(0 ... 0 

1\ 



J = 0 ... 1 

0 



\1 0 

of 



is a symmetric permutation matrix. Note that since JJ' = I, J = J 1 = J'. 
From (7.5.11) it then follows that p = -Jp and £ = J£J. Hence, 


p'S^l = p'(J£J) _ ’l = p'JS’jl = -p'£~'l, 
which implies that it must be zero. Thus, from (7.5.6) we obtain 

0, = (l'£-'Y)/(l'£-'l), 0 2 = (p'S-'Yj/pfS-'p. (7.5.12) 

Further, 

var(0,) = 0|/1'£-*1, var(0 2 ) = 0|/p'£-'p, (7.5.13) 

- A A 

and covffl,, 0 2 ) = 0. So in the case of a symmetric population, the BLUEs of 
0, and 0 2 are always uncorrelated. 

Remark 1. Even though the formulas for 0, and 0 2 given in (7.5.12) are 
much simpler than those in the general case, they still depend on £ -1 . 
However, in the symmetric case the problem of finding the inverse of £ can 
be reduced to inverting an associated matrix having only half the dimension 
of £. The technique exploits the fact that j n = tr ; = a n _ J+i n _ i+l:n 
and the special form of 0, and 0 2 . See Balakrishnan, Chan, and Balasubra- 
manian (1992) for further details. 

Remark 2. A question of interest is whether Y„, which is a linear 
function of order statistics, can be the BLUE of the location parameter 0,. 
For a symmetric parent this is possible only when l'£ -1 = T or £1 = 1. In 
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other words, when = 1 for all i. From Theorem 4.9.1 we have 

learned that this holds when F 0 is standard normal. _What happens with 
skewed distributions? Bondesson (1976) has shown that Y n is the BLUE of 0, 
for all n if and only if F 0 is either standard normal or a gamma translated to 
have mean 0. In all other cases Y n is less efficient than 0 t . 

Remark 3. In the symmetric case, we have noted that 0, and 0, are 

. A A * _ 

uncorrelated. Is this possible when F 0 is skewed? co v(0,,0 2 ) is zero if and 
only if p,'S _1 l = 0. In that case, 0, and 0 2 are given by (7.5.12), as in the 
symmetric case. Now for a gamma distribution translated to have mean 0, 
ji'l = 0, and since Y„ is the BLUE of 0, v £"*1 = 1. Thus we would have 
p,'£ _1 l = fi'l = 0, indicating the fact that 0 t , 0 2 are uncorrelated when F 0 is 
a gamma distribution translated to have zero mean. 

Remark 4. In general least-squares theory the parameter 6 is unre- 
stricted, whereas here 0 2 is constrained to be positive. One natural question 
then is, is it true that 0 2 > 0 with probability 1? To our knowledge this 
question appears to be still open. However, all the available empirical 
evidence supports the conjecture that 0 2 > 0. 

Remark 5. The discussion so far has concentrated on data consisting of 
the full set of order statistics. If the observed data consists of a fixed subset 
of order statistics to be labeled Y, the general formulas given in (7.5.6) and 
(7.5.8)— (7.5.10) continue to hold. This means we can use them to obtain the 
BLUEs and their moments when we have a Type II censored sample. The 
formulas for the symmetric case hold whenever (7.5.11) is satisfied. This 
occurs, for example, when we have a Type II censored sample from a 
symmetric population where the censoring is also symmetric. Finally, note 
that formulas developed here do not yield the BLUEs when the sample is 
Type I censored. (Why?) 


Examples of BLUEs 

The formulas for 0„ 0 2 , and their variances depend on £ -1 , the inverse of 
the covariance matrix of observed order statistics. In general, finding the 
inverse of a large matrix with nonzero entries is a messy affair. But £ is a 
patterned matrix for several distributions. Asymptotically (as n -» ») the 
covariance matrix of a finite number of selected order statistics is a similar 
patterned matrix. We begin with a lemma which is helpful in determining 
£ -1 in such cases. 

Lemma 7.5.1. Let C = (c, 7 ) be a k X k nonsingular symmetric matrix 
with c,j = aft, i <. j. Then C -1 is a symmetric matrix, and for i <,j, its 
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(i, /)th element is given by 


(a 1+1 b, Ojb i+ 1) 


a /+I^i-I a i-I^i + I 


a ; — i^i )( + i^i a i^/+i) 


a 2 {ai(a 2 ^i “ 01*2)} 

bk i[bk( a kbk-i ~ a k-ibk)} > 


7=1 + 1 and 
1 = 1 to k - 1 , 

i = 7 = 2 to A: - 1, 

1=7 = 1 , 

i=j = k, 

7 > i + 1 . 


The lemma follows by the direct manipulation of the fact that CC _1 = I. 
Graybill (1983, pp. 198) calls C -1 a diagonal matrix of Type 2, and has 
another version of the lemma. 


Example 7.5.1 (two-parameter uniform distribution). Let /(y; 0 ,, 0 2 ) = 
0 2 0, - (6 2 / 2) < y < 0, + (0 2 / 2) so that f 0 (x) = 1, - x < |. Sup- 

pose we are interested in the BLUEs of 0! and 0 2 based on the complete 
sample. Examples 2.2.1 and 2.3.1 discuss the moments of order statistics from 
the standard uniform distribution. From them it follows that 


Vi-.n = “ ~ + ~T 

2 n + 1 


and 


a- 


i(n -7 + I) 


,J:n (n + l) 2 (n + 2 ) 


i <}■ 


Thus, Lemma 7.5.1 is applicable to X. Using it, we obtain for i < 7 , 


or 


1 


(« + !)(« + 2) 


(n + l) 
2 


, 7 = i + 1 , i = 1 to n — 1 , 


7=1 = 1 to n , 


(n + l)’ 

0 , j > i + 1 , 


where or'-' is the O', /)th element of X ‘. Since / 0 is symmetric around 0 , 
0j,0 2 are uncorrelated and are given by (7.5.12). On simplification, we get 

0. = \{Yun + Y n:n ), §2 = r ^z\(Y n . n - Y l:n ), (7.5.14) 



176 


ORDER STATISTICS IN STATISTICAL INFERENCE 


and from (7.5.13) we obtain 


var 


(*>) = 


e 2 2 


2 (n + l)(n + 2) ’ 


vari 


<«*) = 


2d\ 

(n + 2)(n - 1) ’ 


Thus, the BLUE of the location parameter is the sample midrange and that 
of the scale parameter is a multiple of the sample range. It is known that 
Y\. n and Y n .„ are jointly sufficient for (0,, 8 2 ) and that (Y i:n ,Y n . n ) is 
complete. Hence, the BLUEs given by (7.5.14) are in fact MVUEs; see 
Sarhan and Greenberg (1959) for more details. 

In the presence of censoring, 2 -1 is a bit more complicated. So are the 
estimators and their moments. For details see Exercise 26. 

Example 7.5.2 (two-parameter exponential distribution). Let us now 
take /(y;0„0 2 ) = 0 2 1 exp(-(y - 0,)/0 2 ), y > 0,. From (4.6.7) and (4.6.8) 
we recall 


ij'.n I . n 


52 (« “ r + 1) 2 , i <j. 

r= 1 


Hence, in Lemma 7.5.1 we can take a, = a; , . „ and by = 1. Once again, using 
the complete sample for simplicity in our calculations, we obtain 


a 


U = 


f(n-/) 2 , ; = / + 1, i = 1 to n - 1 , 

i (n - i) 2 + («-/ + l) 2 , i — j — 1 to n, 

VO, ;>/ + 1. 


Now recall (4.6.6) to note that fi i:n = E‘ r -i (« - r + 1) On substituting 
these values in (7.5.6), we obtain 


(nY Un -Y„) 


n(Y n ~Y i:n ) 


6 1 '(«- I) '’ 62 («-D 


Further, from (7.5.8)— (7.5. 10), it follows that 

0 2 


e 2 2 


e\ 


n(n - 1) ’ 


u 2 (* A \ 2 

— , and ««*(*■•»■) 


™r(#,) - 


var(0 2 ) = 
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In the censored case the details are messy even though the procedure is 
fairly routine. Suppose the data consists of Y r:n ,...,Y s:n where 1 < r < s < 
n. On carrying through the simplifications, one obtains 

#1 = Y r :n- V-r.nBz and 

h = J^- r { i Y i:n - (n - r + 1 )Y r:n + (n - s)Y i;n } . (7.5.15) 

Further, 

v ar (fl,) = + var(fl 2 ) = 0f/(s - r), 

and covCfl„fl 2 ) = -elfi r . n /(s - r). 

Some details about these calculations are available in Balakrishnan and 
Cohen (1991, pp. 88-90). Sarhan and Greenberg (1957) have prepared tables 
which give the coefficients for finding the BLUEs, and the variances and 
covariance of the estimators, for all choices of censoring up to sample size 10. 

One can compare the relative efficiency of these estimators in the cen- 
sored-sample case with that corresponding to the full-sample case. Note that 
the relative efficiencies, the ratios of the variances, are free of the unknown 
parameters. 

Example 7.5.3 (the normal distribution). Since the normal distribution is 
a symmetric distribution, formulas for the BLUEs of 0, (mean) and 0 2 
(standard deviation) are given by (7.5.12). In the uncensored case we have 
seen that 0, is in fact the sample mean. Otherwise, the coefficients of Y i n 's 
have to be evaluated for each choice of r, s, and n. In a series of papers in 
the 1950s Sarhan and Greenberg tabulated these coefficients as well as the 
variances and covariances of the estimators for sample sizes up to 20. They 
are reported in the convenient reference, Sarhan and Greenberg (1962b, 
pp. 218-268). Recently, Balakrishnan (1990) has extended these tables to 
cover sample sizes up to 40. 

For the purpose of illustration, we take the sample size n — 5, and present 
Table 7.5.1 and Table 7.5.2 extracted, respectively, from Table 10C.1 and 
Table 10C.2 in Sarhan and Greenberg (1962b). The original source of our 
tables, Sarhan and Greenberg (1956), reports the values of the coefficients 
and the second moments to eight decimal places. Table 7.5.1 gives the 
coefficients of order statistics for finding the BLUEs of fi and a based on the 
censored sample Y r , 5 , . . .,Y s:S , for r = 1, s = 2 to 5, and r = 2, s = 3,4. 
Since the standard normal pdf is symmetric around 0, the coefficient of Y i n 
for the BLUE of the parameters based on Y, = (Y r . „,..., Y^ ;n ) is related to 
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Table 7.5.1. The Coefficients of Y i:S for the BLUEs of the 
Normal Population Parameters 


Coefficient of 


r 

s 

Est. 

Yl:S 

f 2:5 

F 3:5 

Y *. 5 

^5:5 

i 

2 

A 

-0.7411 

1.7411 






& 

-1.4971 

1.4971 




i 

3 

A 

-0.0638 

0.1498 

0.9139 





& 

-0.7696 

-0.2121 

0.9817 



i 

4 

A 

0.1252 

0.1830 

0.2147 

0.4771 




a 

-0.5117 

-0.1668 

0.0274 

0.6511 


i 

5 

A 

0.2 

0.2 

0.2 

0.2 

0.2 



& 

-0.3724 

-0.1352 

0 

0.1352 

0.3724 

2 

3 

A 


0 

1 





a 


-2.0201 

2.0201 



2 

4 

A 


0.3893 

0.2214 

0.3893 




& 


-1.0101 

0 

1.0101 



Table adapted from Sarhan and Greenberg (1956, Ann. Math. Statist. 27, 427-451). Produced 
with permission of the Institute of Mathematical Statistics. 

the coefficient of Y n _ i+l:n when the data is made up of Y 2 = 
(Y n - s+l:n ,...,Y n _ r+l . n ). In the case of p, the coefficient of Y i:n when the 
given data is Y, and the coefficient of Y„ based on the data Y 2 are the 
same, whereas in the case of &, they differ only in their sign. Thus, Table 

7.5.1 can be used to find the coefficients of Y i;n for determining p or a for 
all censored samples of the form Y r . s ,...,y^. s , where 1 ^ r < s ^ 5. Table 

7.5.2 exhibits the variances and covariances of the BLUEs. Here again the 
symmetry of the normal pdf can be used to read these moments for all 
censored samples of the form Y r . 5 ,...,Y s:S . 

Along with their rather extensive tables, Sarhan and Greenberg (1962b) 
note several interesting facts. For example, they point out that when the 
sample size is odd and only the sample median and its neighbor on either 
side are available, the BLUE is the sample median (Do we see this in our 


Table 7.5.2. Variances and Covariances of p. and a for the Censored Sample 
Y r:S , . . . , Y s;S from N(p.,«r 2 ) when a = 1 


r 

s 

2 

3 

4 

5 


V(p) 

0.6112 

0.2839 

0.2177 

0.2 

i 

V(&) 

0.6957 

0.3181 

0.1948 

0.1333 


co\(p, &) 

0.4749 

0.1234 

0.0330 

0 


V(p) 


0.2868 

0.2258 


2 

V(&) 


0.7406 

0.3297 



cavip,&) 


0.1584 

0 



Table adapted from Sarhan and Greenberg (1956, Ann. Math. Statist. 27, 427-451). Produced 
with permission of the Institute of Mathematical Statistics. 
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Table 7.5.1?) They also observe that the relative efficiency of fi and & in the 
censored case in relation to the full sample case is reasonably high as long as 
the middle observations are available. 

The BLUEs of 0, and 0 2 can be used to obtain the BLUE of any linear 
function of these parameters. It is known from general least-squares theory 
[see, for example, Rao (1973, pp. 223)] that the BLUE of l'0 (= /,0, + l 2 d 2 ) 
is 1'0 where 0, and 0 2 are given in (7.5.6). Further, its variance is 1' cov(0)l = 
0 2 r(A'2~ 1 A) _1 l. Important examples of linear parametric functions are the 
pth population quantile F~'(p ) and the quantile difference F~ l (p 2 ) - 
The BLUE of F~\p) = 0, + 0 2 F o ~'(p) is (-pi + F£ l (p)V)rY, 
where F 0 - '(/>), the pth quantile of the standardized distribution is known. 

Tables of the coefficients of the BLUEs of the location and scale parame- 
ters are available for several common distributions. For details about the 
references, see Balakrishnan and Cohen (1991, pp. 120). 


Asymptotic Approaches to Best Linear Unbiased Estimation 

We have already seen in Section 5.5 through (5.5.4)-(5.5.6) that for large n. 


P,Qj 




(n + 2)f{F~\ Pi ))f(F~\ Pj )Y 


i<j, (7.5.16) 


with p, = 1 — = i/(n + 1). From (7.5.16) it is clear that the covariance 

matrix X of the selected order statistics can be approximated by a patterned 
matrix having the form of C of Lemma 7.5.1. Blom (1958, 1962) exploited this 
feature to obtain “unbiased, nearly best linear estimators” for linear func- 
tions of the location and scale parameters. 

The asymptotic theory of linear functions of order statistics (to be formally 
developed in Section 8.6) has also been used successfully to obtain asymptoti- 
cally BLUEs given a complete or a censored sample. For example, it would 
be nice if we could find a function J(u\ 0 < u < 1, such that 0, is approxi- 
mately n _1 E" =1 7(i/(n + 1 ))Y in . This would eliminate the need for 
extensive tables of coefficients for various (especially large) sample sizes. 
Pioneering work in this direction was carried out by Bennett (1952) and was 
pursued by Chernoff, Gastwirth, and Johns (1967). For a M0,,0 2 ) popula- 
tion, their results indicate, as anticipated, that Y n [for which J(u) = 1] is an 
efficient estimator of 0,. Further, the linear function with J(u ) = <J> -1 (u) 
provides an asymptotically BLUE of 0 2 , and is very close to its finite sample 
BLUE. 

An area of substantial research is in the optimal selection of a predeter- 
mined number of order statistics with the goal of minimizing the variance of 
the BLUE of the parameter of interest. When both the location parameter 0, 
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and the scale parameter 0 2 are unknown,^ the goal is to minimize the 
determinant of the covariance matrix of < [0^0 2 ), which is known as the 
generalized variance of 8. Suppose, for a given sample size n, cost or other 

considerations force us to observe only k order statistics Y i{ . n , Y i2 . H Y) 

where 1 ^ i x < • • • < i k <, n. Then, we can choose them so that 
vaK^Jvar^) ~ cov 2 (0j, 0 2 ) is the smallest possible where the value of the 
generalized variance can be computed from (7.5 .8)— (7.5 . 1 0) for all possible j 
subsets of the vector of all order statistics. One can use the asymptotic joint 
normality of central order statistics (to be developed in Section 8.5) to 
produce optimal choices of 0 < p x < • • • <p k < 1 , where i r = [np r \ + 1 , 
1 < r <, k. This approach was initiated by Ogawa (1951). We have already 
noted in (7.5.16) the structure of the covariance matrix £ of the selected 
order statistics. This helps considerably in simplifying the optimization prob- 
lem. The details are quite involved and beyond the scope of this book. They 
can be found in Section 7.3 of Bailakrishnan and Cohen (1991) or in David 
(1981, Section 7.6). 


Simple Least-Squares Estimators 

Having discussed the BLUEs in detail, it is clear that the computation 
requires inverting £, the covariance matrix of the observed order statistics. 
To avoid this hassle, one can use the simple least-squares estimators, which 
minimize (Y - AflXfY - A0) instead of the BLUEs, which minimize (7.5.5). 
This was suggested by Gupta (1952). Thus, one obtains his estimators by 
replacing £ by I in (7.5.6). On doing this, the simple least-squares estimators 
can be expressed as 

0? = {(pV)l' - (m/Z)|i'}Y/A* and 0$ = (mu' - (m/Z)l'}Y/A* 

(7.5.17) 

with A* = m'L i (p i:n - p) 2 , where p is the average of the p i :n ’s correspond- 
ing to the m selected order statistics. 

While the simple least-squares estimators depend only on the means of 
the order statistics from the standardized distributions, their efficiency (in 
relation to BLUEs) is high, especially for the normal parent. 


7.6. PREDICTION OF ORDER STATISTICS 

Prediction problems come up naturally in several real-life situations. They 
can be broadly classified under two categories: (i) the random variable to be 
predicted comes from the same experiment so that it may be correlated with 
the observed data, (ii) it comes from an independent future experiment. In 
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connection with order statistics, both of these situations are feasible. Let us 
look at a couple of examples. 

Suppose a machine consists of n components and fails whenever k of 
these components fail. Observations consist of the first r failure times, and 
the goal is to predict the failure time of the machine. Assuming the compo- 
nents’ life lengths are i.i.d., we have a prediction problem involving a Type II 
censored sample, and it falls into category (i). One can think of a point 
predictor or an interval predictor for the fcth order statistic. Let us call this a 
one-sample problem. 

A manufacturer of certain equipment is interested in setting up a warranty 
for the equipment in a lot being sent out to the market. Using the informa- 
tion based on a small sample, possibly censored, the goal is to predict and set 
a lower prediction limit for the weakest item in the lot. This falls into 
category (ii), and we call this a two-sample problem. 

Let us now consider the one-sample problem more closely. The best 
unbiased predictor of Y k . „ is E(Y k . „ | Y x Y r . „) = E(Y k . „ | Y r .„) by the 
Markov property of order statistics, assuming, of course, that F is absolutely 
continuous. But when the parameters of F(y\ 6) are unknown, they have to 
be estimated. In the location and scale family case, the best linear unbiased 
predictor (BLUP) can be obtained by using the results on the general linear 
model. It turns out the BLUP of Y k . n is 

Y k -.n = *1 + F k: J 2 + »'S-'(Y - 0,1 - 0 2l i) (7.6.1) 

where 0,,0 2 are the BLUEs of 0, and 0 2 given by (7.5.6) and <a' = 
(°"i ,k:n> ■ ■ ■ ’ a r,k:n)- When a- . „ = a,b ; , i <j, using Lemma 7.5.1 it can be 
shown (please try!) that w'2 - ' = (0, . . . , 0, b r /b k ) and, hence, 

Yk-.n = e, + Hk J 2 + y (Y r:n - 0, - HrJ 2 ). (7.6.2) 


Example 7.6. 1 (two-parameter exponential distribution). When K, . n , . . . , 
Y r:n are observed, from (7.5.15) we have B l = Y l:n -d 2 /n and 0 2 = 
- T 1:n ) + (n - r\Y r n - Y X n ))/(r - 1). Further, since b r /b k = 
1, from (7.6.2) we conclude 


Y k :n Y r:n + (Hk-.n Fr-.n )® 2 ~ Y r . „ + 


' k 

E 

i i = r+) 


i + 1 


0 2 (7.6.3) 


is the BLUP of Y k:n . 
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As in the case of estimation, minimizing the mean-squared error of 
prediction and insisting on unbiasedness are not the only optimality criteria 
statisticians have come up with. Predictors based on Pitman’s nearness 
criterion and on maximizing the likelihood have been discussed in the 
literature and have been applied to the prediction of order statistics. [See 
Nagaraja (1986c), and Kaminsky and Rhodin (1985) for some details.] All of 
these refer to the problem of point prediction. One can establish a prediction 
interval for Y k . „ as well. Let us look again at the two-parameter exponential 
example. 


Example 7.6.1 (continued). Note that, with Z- s being i.i.d. Exp(l), 
(r - 1)0 2 = E [„ 2 (n - * + ixy; :n (recall Theorem 

4.6.1) and, hence, (r - 1 )0 2 /0 2 is T(r - 1, 1). Further, 


Y k:n -Y r:n ±0 2 £ 


, -7 + i «-«'+! 


Z. = Y, 


k—r:n—r‘ 


Since Z,’s are i.i.d. Exp(l) random variables, (Y k:n - Y r:n ) and 0 2 are 
independent. Hence, one can use the pivotal quantity T = (Y k:n - Y r n )/ 
( r - 1)0 2 , where the numerator is distributed like an order statistic from a 
standard exponential distribution and the denominator is a gamma random 
variable and is independent of the numerator. On conditioning with respect 
to (r - 1)8 2 , and using the rule of total probability, we obtain the survival 
function of T to be [see Like§ (1974)] 


P(T> t) 


1 

B(n - k + 1, k - r) 


i=r+l ' 


i) ! ! . 

1 In - i + 1 {1 + (n - i + l)f} r 1 


(7.6.4) 


If P(T > t Q ) = 1 - a, then a 100 a% prediction interval for Y k n would be 
(Y r:n ,Y r:n + t 0 (r-l)B 2 ). 

One can extend the discussion of prediction in a single sample to the 
two-sample problem. Now the goal is to predict Y k;m , the kth order statistic 
of a future random sample of size m from F(y;0) based on the data 
consisting of Y }:n ,...Y r . n . The BLUP of Y k m would be Y k:m = 0, + 
fi k . m 8 2 , si nce covflt^, ^ ;n ) = 0. One can also consider the problem of 
finding a prediction interval for Y k:m . In the case of the two-parameter 
exponential distribution the pivotal quantity T' = (Y k . m - Y i .„)/(r - 1)8 2 
behaves in a manner very similar to the random variable T in Example 7.6.1 
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and may be used to construct a prediction interval. See Lawless (1977) for 
details about the prediction intervals for Y' k . m and also for Y' m , the mean of a 
future independent sample. 


7.7. DISTRIBUTION-FREE CONFIDENCE AND 
TOLERANCE INTERVALS 

We have seen in (2.4.1) that if Y v ...,Y n is a random sample from an 
absolutely continuous cdf F(y;6), F(Y i: „;6) = U i:n , and in (2.5.21) we no- 
ticed that the quasirange U j:n - U i:n = f/ ; _ i:n . The first fact can be used to 
produce confidence intervals for a population quantile F~\p ) whose end- 
points are order statistics and have coverage probabilities free of F, as long 
as it is continuous. On combining the two facts, we can obtain tolerance 
intervals (to be defined later) which do not depend on F. Such procedures 
are known as distribution-free procedures. They can be used when either 6 
or the form of F itself is unknown. 


Confidence Intervals for Population Quantiles 

When F is absolutely continuous, F(F~ t (p)) = p and, hence, we have 

P{Y i:n <F l {p)) = P(F(Y i:n ) <p) 

= P(U i:n <p) 

= L{ n r )p r ( 1 ~P)"~ r (7.7.1) 

r = i 

from (2.2.13). Now for i < j, consider 

P{y, n * F-\p )) = P(Y i:n < F-\p ), Y ]:n < F-\p )) 

+ P{Y in <F-\p),Y j:n >F-\p)) 

= P(Y J:n < F~\p )) + P(Y i:n < F-\p) < Y j:n ). 

Since Y j:n is absolutely continuous, this equation can be written as 

P(Y i:n < F~\p) < Y j:n ) = P(Y i .„ < F-'(P)) - P(Y j:n < F'\p )) 

= L( n r )p r (l-P)"~ r , (7.7.2) 


where the last equality follows from (7.7.1). Thus, we have a confidence 
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interval [ 1^- . „ , ly . „ ] for F~\p) whose confidence coefficient a(i,j), given by 
(7.7.2), is free of F and can be read from the table of binomial probabilities. 
If p and the desired confidence level a 0 are specified, we choose i and j so 
that a(i,j) exceeds a 0 . Because of the fact that a(i,j) is a step function, 
usually the interval we obtain tends to be conservative. Further, the choice of 
i and j is not unique, and the choice which makes (j - i ) small appears 
reasonable. (Why don’t we consider minimizing the expected length of the 
confidence interval?) For a given n and p, the binomial pmf jp r (l - p) n ~ r 
increases as r increases up to around [tip], the greatest integer not exceeding 
np, and then decreases. So if we want to make (j - i ) small, we have to start 
with i and j close to [np] and gradually increase (j - i) until a(i,j) exceeds 
a 0 . In the case of p = 0.5, we obtain the smallest (j - i ) by choosing 
j = n - i + 1. 

Even when the sample size is moderately large, one can use the normal 
approximation (with continuity correction) to the binomial probabilities to 
approximate the confidence level as 


l j - 0.5 - np\ l i - 0.5 - np \ 

}/np( 1 -p) ) \ ~P)Y 


(7.7.3) 


and fo r a given a„ one can choose i = [np + 0. 5 - <& *((1 + 
a 0 )/2)yjnp{\ - p) ] and j = [np + 0.5 + <I>~ 1 ((1 + a 0 )/2)^np(l - p) ] to 
obtain a confidence interval with an approximate confidence level of a 0 . 

One can also construct parametric confidence intervals for F~ l (p) which 
assume the form of F to be known. Such intervals constructed from sufficient 
statistics probably do better when our assumptions regarding F are valid, but 
obviously are not as robust as the distribution-free procedure we have 
discussed. We are now ready for an example. 

Example 7.7.1. The following 10 observations, simulated from the stan- 
dard normal distribution using the statistical package MINITAB, are pre- 
sented in increasing order: -1.61, -1.36, -0.76, -0.57, 0.05, 0.27, 0.32, 
0.60, 0.82, and 1.26. For this data the sample mean is -0.098, and the sample 
standard deviation is 0.946. The 89.06% confidence interval based on the t 
distribution for the population mean, which is also the population median, is 
(-0.629,0.433). Using tables for the Bin(n, p) pmf with n = 10 and p = 0.5, 
we see that (T 3; i 0 , F 8:10 ) provides a confidence interval for the population 
median with the same level of confidence; with this data the interval is 
( - 0.76, 0.60). The approximate confidence level of this interval as given by 
(7.7.3) turns out to be 88.58%. Don’t expect the normal approximation to be 
this good if p is far away from 0.5. If our data had actually come from a 
Cauchy population, our parametric interval would be questionable, while the 
one based on order statistics would still be valid! 
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Even though in the discrete case (7.7.2) does not hold, we will be very 
close; in fact the confidence level of [Y i:n ,Y j:n ] is no less than a(i,j), 
whereas the confidence level of ( Y i:n ,Y j:n ) is no more than a(i,j). (Can you 
show?) 

The technique developed here for a single quantile can be extended to 
obtain conservative confidence intervals for quantile differences. This could 
be handy if one is interested, for example, in the interquartile range. Some 
details may be found in David (1981, pp. 15—16) and references therein. The 
earliest work in this area seems to be that of Thompson (1936). 


Tolerance Limits and Intervals 

Consider a population with absolutely continuous cdf F. For an interval 
(a, b), the proportion of the population falling in that interval is F(b) - F(a). 
Now suppose L, and L 2 are random variables such that L, <L 2 with 
probability 1; then the random interval (L,, L 2 ) contains the random propor- 
tion F(L 2 ) - F(L,) of the population. Suppose these L, and L 2 are chosen 
such that 


P(F(L 2 ) — F(L { ) >/?)=a; (7.7.4) 

the interval (L,, L 2 ) will contain at least 100/3% of the population values 
with probability a. This interval is called a 100/3% tolerance interval at 
probability level a, and its endpoints are called the tolerance limits. The 
concept of tolerance interval, originally due to Shewhart (1931), has found 
applications in quality control. Quite often one takes a and /3 to be either 
0.95 or 0.99. Order statistics come into the picture here in the form of L , and 
L 2 . Let us now see how. 

Suppose we select a random sample Y l ,...,Y n from this population. On 
taking L, = Y i n and L 2 = Y j:n where i < j, we have L, < L 2 with probabil- 
ity 1 and 

F(L 2 ) — F(L,) = F(Yj. n ) - F(Y j:n ) 

= Uj:n - U,:n 

with the last equality following from (2.5.21). Hence, in (7.7.4) we have 

-'if(?)r( i-or' 

r = 0 

= say. 


(7.7.5) 
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This probability depends only on (/ - /) and is free of F. In other words, the 
interval (Y i:n ,Yj. n ) forms a distribution-free 1000% tolerance interval for the 
population at probability level a*_, given by (7.7.5). Now if a is specified, we 
choose some /, j such that a*_ i exceeds a, and this results in a conservative 
interval. For specified a and 0 one may have to take sufficiently large 
n so that there exist i, / for which this is possible. From (7.7.5) it is clear 
that a *_ , increases as (/ - i) increases and, hence, its maximum value is 
1 - n0" _, (l - 0) - 0" (How?). This monotonically increases to 1 as n 
increases and, thus, ultimately exceeds a for any given a < 1. One can solve 
for the threshold value of n. 

Sometimes one-sided tolerance regions will be of interest. For example, 
based on the life lengths of a sample of gadgets from a lot, we may want to 
set up an interval of the form (L,, <») with the following property: At least 
90% of the untested gadgets in the lot have life lengths exceeding L, with 
probability of, say, 95%. We can easily modify the above discussion to obtain 
the lower tolerance limit L,. Note that P( 1 - F(X i n ) > 0) = a*_ i+l , where 
a* is given by (7.7.5). Thus, the interval (X i:n ,») contains at least 1000% of 
the population values with probability 


7.8. GOODNESS-OF-FIT TESTS 

In many nonregular cases (in the sense of Section 7.3) the order statistics are 
minimal sufficient and play a crucial role in the development of optimal tests. 
Even in regular cases, tests based on order statistics abound in the literature. 
For example, they are used for testing goodness-of-fit of distributions like the 
exponential and uniform, and for testing for outliers. The goodness-of-fit 
tests are also available when the data is a censored sample. Instead of 
squeezing in all these rather specialized topics in our elementary discussion, 
we work with some special cases only with the goal of providing a feel for this 
vast area of continuing research. 


Graphical Procedures 

We will begin with a graphical, rather informal, method of testing the 
goodness-of-fit of a hypothesized distribution to given data. This procedure, 
known as Q-Q pbt, was introduced by Wilk and Gnanadesikan (1968). It 
essentially plots the quantile function of one cdf against that of another cdf. 
When the latter cdf is the empirical cdf defined below, order statistics come 
into the picture. Such Q-Q plots are widely used in checking whether the 
random errors in an assumed linear regression model behave like a random 
sample from a normal distribution. Statisticians routinely produce these plots 
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as part of their data-analysis diagnostics. Let us formally describe the 
procedure in the context of order statistics. 

As in Section 7.5, let Y l ,Y 2 ,...,Y n be a random sample from the 
absolutely continuous cdf F(y,B l ,6 2 ), where 0, is the location parameter 
and 6 2 > 0 is the scale parameter. The empirical cdf, to be denoted by F„(y) 
for all real y, represents the proportion of sample values that do not exceed 
y. It has jumps of magnitude l/n at Y i;n , 1 < i < n. Thus, the order statistics 
represent the values taken by F~'(p), the sample quantile function. (See 
Exercise 34 for some further details about F n .) The Q-Q plot is the graphical 
representation of the points ( F~ l (Pj),Y i:n ), where population quantiles are 
recorded along the horizontal axis and the sample quantiles on the vertical 
axis. There are several suggestions about choosing the p,’s. Usually they are 
of the form p, = O' - c)/(n -2 c + 1), where 0 < c < 1. With c = 0, one 
obtains p, = E(U i n ), and with c = 0.3175, p, will be close to the median of 
the distribution of U i: „. Note that F~\p) = 0, + 6 2 Fq'(p) where F 0 , the 
standardized cdf, is completely specified. Thus we can plot (Fg l (p,),Y i:n ) 
and expect the plot to be close to a straight line if the sample is in fact from 
F(y;ff l ,0 2 ). If not, the plot may show nonlinearity at the upper or lower 
ends, which may be an indication of the presence of outliers. If the nonlinear- 
ity shows up at other points as well, one could question the validity of the 
assumption that the parent cdf is F(y; 0,, 0 2 ). In that case, can the Q-Q plot 
suggest some plausible alternatives? In some cases yes, as can be seen from 
the interesting discussion in D’Agostino (1986a) about the normal and 
log-normal families. Further, if the plot appears close to a straight line, we 
can estimate 0, and 0 2 using the intercept and the slope of the line of best 
fit. The Q-Q plots can also be used to inspect the closeness of the fit, even 
when the data is censored. 

Plots based on order statistics are also used for checking whether the 
parent pdf is symmetric. The simplest one is the plot of (Y i: „,Y n _ j+I . n ) for 
i = l,.. . ,[n/2], whose slope is always negative. If the symmetry assumption 
holds, we expect a slope of - 1. If the slope is less than - 1, it is an indication 
of positive skewness, and if it exceeds -1, we suspect the distribution to be 
negatively skewed. Wilk and Gnanadesikan (1968) note another scheme 
suggested by J. W. Tukey in which one plots the sums Y i:n + Y„_ i + l .„ 
against the differences Y i:n - Y„_ i+l .„, i = l,...,[n/2]. For a symmetric 
distribution, we expect the configuration to be horizontal. 


Tests for the Exponential Distribution 

Let us consider the problem of testing whether the available Type II 
censored sample Y t . n ,...,Y r n is from the two-parameter exponential pdf 
/(y; 0,, 0 2 ) = (l/02)exp(-[(y - 0,)/0 2 ]), y>0 The likelihood function 
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for this data is given by 

81 zyi < • •• <y r - 


Thus, the MLEs are 

9x-Y l:n and § 2 - l^t(n - i + l)(Y i:n - Y^.^j r. (7.8.1) 

Note that (0,,0 2 ) is the complete sufficient statistic for (0 U 0 2 ) and, hence, 
the MVUEs of the parameters can be determined easily. In fact, the BLUEs 
obtained in Example 7.6.1 are the MVUEs of the respective parameters. 
Under the assumption of exponentiality, n(0 x — 0 x )/0 2 is standard exponen- 
tial, and rd 2 /d 2 is T(r - 1, 1). More interestingly, they are independent 
(Exercise 16). For what other distribution have you encountered such a 
phenomenon where the MLEs of the location and scale parameters are 
independent? 

Suppose we want to test the null hypothesis H 0 : 0, = 6° (a known 
quantity) against the alternative //,: 0, *6°, when 0 2 is unknown. Then the 
likelihood ratio test rejects H 0 if (L(0?, 0\)/L(0 v 0 2 )} is small. Here, 0° is 
the MLE of 0 2 under H 0 , and 0„ 0 2 are the MLEs with no restrictions on 
the parameters, and are given by (7.8.1). This leads to a test based on the 
statistic T, = n(0 l - 0?)/0 2 which has an F i22jLr _ X)) distribution under H 0 . 
One rejects H 0 if T, < 0 or if 7\ is large. On the other hand, if H 0 : 0 2 = 0 2 
and H{. 0 2 # 0 2 , the likelihood ratio test procedure is based on the statistic 
T 2 = 2 r0 2 /0\, which has a * 2 (2(r - 1)) distribution when H 0 is true. 

In a goodness-of-fit testing situation, H 0 says the parent cdf is exponential 
with values for the parameters unspecified. We could use the usual Karl 
Pearson chi-square test if we have a complete sample and the sample size is 
large. We could also use some of the characterization results discussed in 
Chapter 6 to test for exponentiality. In fact, when F is one-parameter 
exponential with the parameter being the scale, a test has been suggested in 
Section 6.1. One can devise any number of test procedures which possess 
good power against a certain type of alternative, and that is exactly what has 
been done in the literature. For example, Epstein (1960a, 1960b) has sug- 
gested one dozen tests for exponentiality. Choosing a test should be based on 
the type of alternatives one has in mind; for example, the alternative may be 
that F has decreasing or increasing failure rate. We do not plan to catalog 
and discuss the large number of tests for exponentiality suggested in the 
literature. Interested readers may consult the extensive survey by Stephens 
(1986). 
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Just as in the case of the exponential distribution, many goodness-of-fit 
tests have been proposed for normal and for uniform distributions. The 
former is an important distribution in its own right, and the latter is 
important because as we all know by now, the probability integral transfor- 
mation reduces a hypothesis on an arbitrary (absolutely) continuous distribu- 
tion to the one based on the uniform distribution. Prominent test procedures 
suggested for this purpose often depend on order statistics or the empirical 
cdf. Even under the null hypotheses, the distributions of the test statistics are 
messy, and special tables, quite often based on simulation, are needed to 
apply these tests. One test for normality suggested by Shapiro and Wilk 
(1965) uses the ratio of the BLUE of the population standard deviation 
discussed in Example 7.5.3 and the sample standard deviation as the test 
statistic. We do not pursue the details here. Instead we refer to an elaborate 
review of tests for normality by D’Agostino (1986b). 


7.9. OTHER APPLICATIONS 

Our basic assumption throughout this chapter (in fact in this book) is that the 
sample is a random sample; in other words Y x ,...,Y n are i.i.d. random 
variables with cdf F(y;0), where the parameter 0 is possibly unknown. But 
what if this basic assumption is violated in that one or possibly more of the 
>”s are from a different population having cdf F* which may or may not be 
completely specified? The values generated from F* are labeled outliers or 
discordant observations in the literature. The possible existence of these 
outliers raises several natural questions about the decision rules we propose 
on the basis of our assumptions about the sample. One of them is how to test 
for the existence of outliers and identify them, and another is how to devise 
decision rules which are less influenced' by these outliers. We will briefly 
discuss these topics here. Another area of voluminous research is that of 
ranking and selection of populations where the basic assumption is that the 
>?s, while being independent, come from distinct populations. The goal may 
be to rank the populations or select the best population based on our data. 
The discussion of these topics is beyond the scope of this text. Interested 
readers may consult either Gibbons, Olkin, and Sobel (1977) or Gupta and 
Panchapakesan (1979) for an introduction. Another related area where order 
statistics play a major role is that of multiple comparison procedures. They 
are discussed in Hochberg and Tamhane (1987). There again, the basic 
premise is that the Y t 's are nonidentically distributed. 


Tests for Outliers 

When one suspects the presence of outliers in the sample, there are interest- 
ing statistical issues to deal with, some of which are: (i) How to test for the 
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existence of outliers so that our modeling of the data can be validated? (ii) If 
we accept the hypothesis that outliers are present, how can we identify them? 
(iii) If one or more of the extreme order statistics are too far away from the 
rest, should we throw them out because of the suspicion that they are 
outliers? (iv) What are the implications of our tests for outliers on subse- 
quent statistical procedures? In this maze of questions, there are no simple 
or obvious answers. Let us now consider a simple problem involving the 
normal population for the purpose of illustration. 

Let H 0 : Y x Y„ be a random sample from N(p, a 2 ) distribution and 

H x : One of these YJ’s is from Nip. + 8, a 2 ), where 8 > 0, and the rest are 
from Nip, a 2 ). The actual values of p, 8, and a 2 are unknown. Here, F is 
the Nip, a 2 ) cdf, while F* is the cdf of a Nip + 8, a 2 ) population. This 
alternative hypothesis is known as the slippage alternative in the sense that 
one of the sample values is from a different population (with cdf F*), whose 
location parameter has slipped (to the right). Another commonly used 
alternative is the mixture alternative, where the alternative states that the Y”s 
come from the mixture cdf (1 - p)F + pF* for some p, 0 < p < 1, but we 
will stick with the slippage alternative in what follows. 

What are the performance measures which are suitable for comparing test 
procedures in this situation? If our goal is just to know whether there is 
evidence to reject H 0 and we are not concerned with finding out which 
observation is the outlier when we reject it, we would try to maximize the 
power function /‘(Rejecting H 0 \H { ). But life is not that simple. We may 
need the noncontaminated observations to produce an estimator of ft, in 
which case we need to know which one to throw out as an outlier if H 0 is 
rejected. Since when H x is true, P(Y i n is the outlier) is maximized when 
i = n, we can construct a test based on the statistic 

T={Y n;n -Y n )/S, (7.9.1) 

where S 2 = - Y n ) 2 /in - 1) is the sample variance. The test proce- 

dure rejects H 0 if T is large. If H 0 is rejected, we remove Y„.„ from the 
sample or give less weight to it in subsequent statistical analyses. Barnett and 
Lewis (1984, pp. 167-168) enumerate several optimal properties of this test 
procedure. For example, it is the likelihood ratio test and maximizes the 
probability of correctly identifying the outlier among tests which are location 
and scale invariant. It was first proposed by Pearson and Chandrasekar 
(1936), who essentially compute the percentage points of T for some values 
of n. Note that T, sometimes referred to as the internally studentized 
extreme deviate, belongs to the class of statistics discussed in Exercise 4.17. 

Percentage points of T under H 0 are given in Table 7.9.1 for some small 
values of n. The table was extracted from the tutorial article by Grubbs 
(1969), which contains a nice discussion on outliers. 

Example 7.9.1. Consider the following data, consisting of 15 observa- 
tions generated independently and arranged in increasing order. Using 
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Table 7.9.1. 

Percentiles of the Distribution of (T 

-V/s 

in Random Samples from a Normal Population 

Sample Size 

95th 

97.5th 

99th 

in) 

Percentile 

Percentile 

Percentile 

3 

1.15 

1.15 

1.15 

4 

1.46 

1.48 

1.49 

5 

1.67 

1.71 

1.75 

6 

1.82 

1.89 

1.94 

7 

1.94 

2.02 

2.10 

8 

2.03 

2.13 

2.22 

9 

2.11 

2.21 

2.32 

10 

2.18 

2.29 

2.41 

11 

2.23 

2.36 

2.48 

12 

2.29 

2.41 

2.55 

13 

2.33 

2.46 

2.61 

14 

2.37 

2.51 

2.66 

15 

2.41 

2.55 

2.71 

20 

2.56 

2.71 

2.88 


Table adapted from Grubbs (1969, Technometrics 11, 1-21). Produced with 
permission from Technometrics. Copyright 1969 by the American Statisti- 
cal Association and the American Society for Quality Control. All rights 
reserved. 


MINITAB, 14 of these were simulated from the standard normal population, 
and the remaining one was generated from a M3, 1) distribution: -1.671, 
-1.506, -1.199, -0.759, -0.529, -0.434, -0.145, 0.189, 0.335, 0.591, 
0.592, 0.702, 0.770, 1.141, 3.656. For this sample, the mean is 0.116 and the 
sample standard deviation is 1.308. Hence, the calculated value of the 
statistic T is 2.71 which turns out to be the 99th percentile of the distribution 
of T under the null hypothesis of no outliers. Without the sample maximum, 
which in fact came from the outlier distribution, the sample mean would be 
-0.137 and the sample standard deviation would be 0.900. When the sample 
is made up of these 14 observations, one obtains the value of T to be just 
1.42. 

Let us see how the Q-Q plot looks. We used MINITAB to produce the 
population quantiles <t>~ '(/?,) in which p x , = (i - 0.375)/(n + 0.75). The 
package SYSTAT (1990) was used to produce the following plot of 
J/ :I5 ), i = 1 to 15. 

Tests for outliers are often susceptible to a so-called masking effect, and 
the statistic T given by (7.9.1) is no exception. In this context, the masking 
effect arises, for example, in the presence of two outliers. If in fact there are 
two observations in the sample which are from the slipped population, the 
value of T tends to be smaller and, hence, we may end up accepting H 0 
quite often even when it is false. See Exercise 36 for an illustration. To 
handle such a situation, we can test for the presence of multiple outliers, as 
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Figure 7.1. Q-Q plot of the data in Example 7.9.1. 



was done by Murphy (1951). The test statistic used by him compares the 
average of the top k sample values with the sample mean in terms of sample 
standard deviation units. Such statistics are also known as selection differen- 
tials. (See Example 8.6.5 and Chapter 4, Exercise 17 for some details.) 

The literature on outliers has been well synthesized in the comprehensive 
work by Barnett and Lewis (1984). They treat various aspects of this problem 
in general. For several common distributions, they list test procedures and 
present tables containing percentiles of the various test statistics used. 
Another useful reference on outliers is the monograph by Hawkins (1980). 


Robust Estimators 

In view of the discussion we had about outliers, concerns arise about 
commonly used estimators of parametric functions of interest. Can we find 
estimators which are affected minimally by the presence of outliers? How 
does the estimator which we claim to be optimal when the parent cdf is F 
behave when a part or the whole of the sample is from F*1 Issues like these 
lead us to the study of robust estimators whose properties are less susceptible 
to model violations. The types of violations considered in the literature can 
be broadly classified as being of the following types: (i) The outliers come 
from F*, a cdf related to F through a change of location and/or scale, (ii) 
The entire sample is from an altogether different population with cdf F*. 
Note that the distribution-free confidence interval for the population quan- 
tile constructed in Section 7.7 is robust against the second kind of violation. 
As you can imagine, it is impossible to construct an estimator which is 
optimal under all circumstances. To achieve robustness against the violation 
(i), especially when F* is from a slipped population, one usually omits or 
gives less weight to the sample extremes while constructing the estimator. 
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Sometimes this also achieves robustness against the violation (ii), even 
though the choice of F* there pretty much decides the fate of an estimator 
when the departure from the assumed model takes place. 

One popular robust estimator of the center of a symmetric distribution is 
the symmetric trimmed mean 

Tr=~ZT r Z Y i:n, 0 < P < [(« - l)/2] , (7.9.2) 

n i=r+l 


where we have trimmed the top r and the bottom r order statistics. Note 
that the two extreme values of r yield the sample mean ( r = 0) and the 
sample median (r = [(n - l)/2]) in (7.9.2). Clearly, the trimmed means are 
L statistics. Among such statistics, the ones which give less weight to the 
sample extremes are suggested as robust estimators. They are robust against 
the presence of a moderate number of outliers and are highly efficient when 
there are no outliers. However, their exact distributions are generally ex- 
tremely complicated, and, hence, either extensive simulation or their asymp- 
totic distributions (usually normal) are used to study the properties of such 
estimators. As pointed out earlier, in Section 8.6 we formally introduce L 
statistics and discuss the relevant asymptotic theory. Special attention will be 
given to the trimmed mean. In Example 8.6.4, we will establish the asymp- 
totic normality of T r when r = [ np ], 0 < p < 1, under some mild conditions 
on F. 

Example 7.9.2. Let F V F 2 ,F 3 denote the cdfs of N(6, 1), the Laplace 
distribution with location parameter 6, and the Cauchy distribution with 
location parameter 6, respectively. Note that all these are symmetric distribu- 
tions having bell-shaped pdfs, but the tail thickness^ increases with i, i = 1 to 
3. Consider the two estimators the sample mean Y n and the sample median 
Y n . Which one is a better estimator of 61 The answer depends on the 
assumed form of the cdf. The estimator Y n is the MVUE and the MLE for 6 
when F = F x , but is no better than a single observation and has infinite 
expectation when F = F 3 . In contrast, Y n is unbiased and consistent for 6, 
and is robust against outliers in all the three cases. Besides, it is the MLE of 
6 when F = F 2 . In that case, as we shall see in Example 8.5.1, for large n at 
least, Y n has smaller variance than Y n . 

To achieve increased precision when the model assumptions hold, while 
retaining robustness, adaptive estimators have been proposed. These take 
different functional forms depending on the value of some statistic computed 
from the sample itself. Hogg (1974) provides an interesting overview of 
adaptive robust procedures. 
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Example 7.9.3. Suppose we are sampling from a N(p, 1) population 
where we suspect one of the values is from N(p + S, 1), 5 * 0. To guard 
against this outlier, one could use the adaptive estimator 


T = 



ifmax{(y„ - - y„)| <c, 

otherwise, 


(7.9.3) 


where T r , r = 0, 1 are the trimmed means given by_(7.9.2) and c can be a 
specified percentile of the distribution of max|Yj — y„|. This statistic is used 
for testing for a single extreme outlier, and its properties and percentiles are 
discussed in Barnett and Lewis (1984, pp. 188-189). When there is no outlier, 
T given in (7.9.3) is, in fact, unbiased for p. 

Sophisticated tools have been used to describe and study robust proce- 
dures in general. Huber’s (1981) treatise serves as a useful reference on 
robust statistics. In an extensive simulation study known as the Princeton 
study, Andrews et al. (1972) have investigated the robustness aspects of 
several estimators of location based on order statistics. 


EXERCISES 

1. A deck has N cards labeled 1 through N. The value of N is unknown. 
We draw a random sample of size n cards without replacement and the 
numbers on those cards are noted. 

(a) Show that the maximum number we have observed, T, is a sufficient 
statistic for N. 

(b) Use T to suggest an unbiased estimator of N. 

(c) Determine the variance of your estimator. 

(Hint: Look at Section 3.7.) 

2. In Exercise 1 suppose the sampling was done with replacement. 

(a) Again show that the sample maximum, T, is sufficient for N. 

(b) Show that {T H + i - (T - l)" + 1 }/{7’" - (T - 1)"} is unbiased for N. 

3. Let f(y,0) = g(fl)My), a(0) <,y <, b(0), flefl, where a(0) decreases 
and b(0) increases. Let us assume that g(0) is differentiable and that we 
have a random sample of size n from this pdf. 

(a) Show that T = max(a _1 (y i: „), 6 _1 (y„ : „)) is the sufficient statistic. 

(b) Show that the MVUE of 0 is 0 n = T - g(T)[ng'(T)}- 1 . 
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4. Let /(y; 0) satisfy the conditions of Exercise 3. 

(a) Show that Q(T,8 ) = g(8)/g(T) is a pivotal quantity and determine 
its distribution. 

(b) Use Q(T;8) to obtain a 100a% confidence interval for g(0) of the 
form (a i g(T),a 2 g(T)) and describe how you will determine a, and 
a 2 . 

(c) Choose a, and a 2 such that a 2 - a, is the smallest. The resulting 
interval is the shortest one among the 100a% confidence intervals 
having the form (a x g(T), a 2 g(T)\ 

(d) Suppose our interest is to obtain confidence intervals for l/g(0) 
along the above lines. How would you proceed? 

(Ferentinos, 1990). 

5. (a) Let us take a random sample of size n from the pdf /(y; 8) = 

g(0)/i(y), a <.y <9, where g(8) is differentiable. Let c(0 ) be a 
differentiable function of 6. Show that the MVUE of c(0) is given by 

c(Y n:n ) + c'(Y n:n )/ng(Y n:n )h(Y n .J- 

(b) Using (a) find the MVUEs of the mean and the variance of the 
distribution having pdf /(y; 0) = 2y0 -2 , 0 < y < 0. 


6. From a random sample of size n from Uniform(0, 0), let us suppose the 
sample maximum is missing. 

(a) Show that Y n _ i:n is a complete sufficient statistic for 0. 

(b) Let c(8) be a twice differentiable function of 0. Show that the 
MVUE of c(0) is given by d(Y n _ i:n ) where 


d(y ) = c(y) + 


2yc'(y) 

n — 1 


y 2 c"(y) 
n(n - 1) 


(c) Obtain the MVUE of the variance of the uniform distribution. 


7. Suppose we have a Type II censored sample from Uniform(0, 0) where 
only the first r order statistics are observed. 

(a) Show that 0 = nY r n /r is the MLE of 0. 

(b) If r = [np], 0 < p < 1, show that 0 is asymptotically normal. Identify 
the norming constants. (Use the informal discussion in Section 1.1. A 
rigorous proof of the asymptotic normality of Y r n may be found in 
Theorem 8.5.1.) 


8. Assume we are sampling from the Pareto distribution with shape param- 
eter 0 whose pdf is given by /(y; 0) = 0y -(9+1) , y ^ 1. 

(a) Find the Fisher information measure contained in the right-censored 
sample Y in ,. . . ,Y r:n . 
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(b) Find the MLE of 8 based on the data in (a) and determine its 
limiting distribution as r becomes large. What is its limiting vari- 
ance? 

9. Let Y, Y„ be a random sample from the cdf F(y\ 8), where 8 is the 

location parameter. Find the BLUE of 8 and give an expression for its 
variance. 

10. Let (Y r:n Y s n ) denote the vector of order statistics from a doubly 

censored sample from f(y;8) = e~ (y ~ e \ y ^ 8. 

(a) Determine the MLE of 8. 

(b) Determine the BLUE of 8 and finds its variance. Is it also MVUE? 

(c) What is Gupta’s simple least-squares estimator of 0? 

11. Let f(y,8) = 1, 8 - £ 8 + \. Find the BLUE of 8 based on 

Y = (Y r:n Y s .„) and find its variance. 

12. Suppose f(y\8) =1, 8 £y <, 8 + 1. Show that T n = (Y 1; „ + Y n:n - 
l)/2 is the BLUE of 8. 

For Exercises 13-16 assume that the data is from the cdf F of a 
two-parameter exponential distribution discussed in Example 7.5.2 where 
both the location parameter 8 t and the scale parameter 8 2 are unknown. 

13. (a) Find the quantile function F~\p) for all p, 0 <p < 1. 

(b) Find the BLUE of F~\p) based on all order statistics of a random 
sample of size n from F and compute its variance. 

(c) Another estimator of F~ l (p) is Y i n where i = [np] + 1. Show that 
Y i n is an asymptotically unbiased and consistent estimator of F~\p). 
How does it compare with the BLUE? 

14. Let the data consist of the entire random sample of size n. Show that T c 
is the MVUE of the survival function P(Y > c ) where 


7L = 



n{Y n -Y l:n )) 


if c < Y l:n , 

if Y Un <Zc<iY l:n +n(Y n -Y l:n ), 
ifc> Y 1; „ + /i(Y„-Y 1: „). 


(Laurent, 1963) 
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15. Suppose now we have observed only the first r order statistics of the 
random sample. 

(a) Are the BLUEs of the parameters also MVUEs? 

(b) Find the BLUE of the mean of the distribution. 

(c) What is the MLE of the mean? 

16. Let 8 X , § 2 be, respectively, the MLEs of 0 X , 0 2 based on Y Xn ,...,Y r . n . 
They are given by (7.8.1). 

(a) Show that IF, = n(0 x - 0 x )/0 2 is standard exponential and W 2 = 

r§ 2 /0 2 is T(r - 1, 1). Express W x and W 2 in terms of chi-square 
random variables. 

(b) Show that W x and W 2 are mutually independent. What is the 
distribution of W x /W 2 ? 

(c) Describe how you may use the conclusions of (a) and (b) to obtain 
confidence intervals for 0, and 0 2 . 

17. Suppose we are sampling from the Rayleigh distribution with cdf F(y\0 ) 
= 1 - exp(-(y/0) 2 ), y > 0, 0 > 0. 

(a) Assuming the data is a Type II censored sample Y X n ,...,Y r . n , 
determine 0, the MLE of 0. 

(b) Now suppose we have a Type I censored sample being censored to 
the right at t. What will be 0 for this situation? 

18. Find the MLE of 0 based on a random sample of size n from the 
Laplace distribution whose pdf is given by f(y,0) = ^exp(-|y - 0|), 
— oo < y < oo. is it unique? 

19. (a) Show that F~ l (p) = 0 X + 0 2 F^{p\ where F(y; 0 X , 0 2 ) is the cdf of 
a distribution with location parameter 0 X and scale parameter 0 2 > 0 and 
F 0 (y) = F(y; 0,1). 

(b) Give an expression for the BLUE of F~\p) based on order statistics 
and an expression for its variance when we have a random sample of size 
n from N(0 X ,0\). 

(c) Under the setup described in (b) obtain the BLUE of F~ '(0.75) - 
F _1 (0.25), the interquartile range. 

20. The following five observations are the order statistics of a random 
sample of size 5 generated from the standard normal distribution using 
MINITAB: -1.024, -0.744, -0.156, 0.294, and 0.746. 

(a) Using the data (Y r:5 , . . . , y j:5 ) for all possible r and s such that 
1 <= r < s < 5, find the BLUEs of the population mean and standard 
deviation. 
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(b) When the available data is find the BLUE of the 

60th percentile of the population. What is the variance of your 
estimate? 

21. (a) Simplify (7.5.17) to determine the coefficients of Y j:n explicitly in 
terms of s for computing Gupta’s estimators 0* and 0*. 

(b) Suppose we want to determine these coefficients for censored sam- 
ples from the normal population. Take n = 5 and make a table similar to 
Table 7.5.1 to display the coefficients of Y i[5 when the available data is 

Y r:S Y s . s , for all choices of r and s. Do this for determining both 0* 

and 0*. 

(c) Make a table similar to Table 7.5.2 giving the variances and covari- 
ances of Gupta’s estimators for all the censored samples you consid- 
ered in (b). Recall that Tables 4.9.1 and 4.9.2 give all the necessary 
information. 

(d) Using the table you constructed^ in (c) above, and Table 7.5.2, 
compare the variances of 0* and 0,, for i = 1,2. 

22. Suppose the observed data consist of just two order statistics Y i:n and 
Yj. n , \ <.i <j <,n. 

(a) Show that for the linear model given by (7.5.3) and (7.5.4), 
(A'£ -1 A) -1 A'E -1 = (AA)“‘A'. 

(b) Usin§ (a) or otherwise, show that the BLUEs 0, and Gupta’s estima- 
tors 0* coincide for i = 1,2, for this data. 

23. Let Y denote a subset of the vector of all order statistics from a random 
sample of size n from the cdf F(y; 0), where 0 > 0 is the scale parame- 
ter. Find 0, the BLUE of 0, and give an expression for the variance of 0. 

24. Suppose we have a Type II censored sample Y r . Y s:n from an 
Exp(0) population. 

(a) Let 0 denote the BLUE of 0. Show that 



/*r:, 


-( n-r ) 


S— 1 


^:„ + E ^ :n + (n-s + l)F j;n 

f-r+l } 


and var(0) = 0 2 /K with K = n 2 r . n /<r r r;n + (s - r ). Recall that ii r:n 
and a- r r . „ are given by (4.6.6) and (4.6.7), respectively. 

(b) Calculate 0*, the simple least-squares estimator of 0. 

(Sarhan and Greenberg, 1957) 
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25. What is the BLUE of 0 based on a random sample of size 2 from 
/(y;0) = 30 3 /y 4 , y>0? 

26. From the uniform population considered in Example 7.5.1, suppose we 
have observed the vector of order statistics (Y r .„,...,Y s:n ),l < r < s < n. 

(a) With this data, show that the BLUEs of 0, and 0 2 are given by 

0, = 2( /l r) -{(* " 2r + 1 )Y s:n + (2 s-n- 1 )Y r:n ) and 
* n + 1 

<>2= — {Y s :n~Y r:n ). 

(b) Find the variances of 0, and 0 2 and covffl,, 0 2 ). 

(c) Compare vaK0j) in this censored case with the one in the complete 
sample case.^ The ratio of these variances provides the relative 
efficiency of 0, as a function of r and s. 

(d) Show that the relative efficiency of 0 2 in the censored case when 
compared to the whole sample case is a function of (s - r). 

(e) Simplify 0, when n is odd and r = (n + l)/2. Comment on your 
answer. 

(Sarhan and Greenberg, 1959). 

27. Suppose we are interested in predicting Y k:n based on the Type II 
censored sample Y l . n , . . . , Y r:n taken from a two-parameter uniform 
distribution introduced in Example 7.5.1 (r < k < n). 

(a) Assuming that both the parameters are unknown, find the BLUP of 

Y k :n- 

(b) What is the BLUE of E(Y k:n )‘> 

28. The mean-square error (MSE) of a predictor Y while predicting the 
random variable Y is defined to E( Y - Y) 2 . In the prediction problem of 
Example 7.6.1 obtain the MSE of Y k:n given by (7.6.3). 

29. Verify (7.6.4). 

30. As in Example 7.4.3, suppose we have observed Y l:n ,...,Y r:n from a 
one-parameter exponential distribution with scale parameter 0. The goal 
is to find a prediction interval for Y k . n where r < k < n. Let 

Ti= L Y l:n + (n- r)Y,, n = £ (n - i + 1)(^,„ - ^_ 1:n ) 

;=i i=i 


represent the total time on test. 
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(a) Show that W x = (Y k . n - Y r:n )/T x is a pivotal quantity whose sur- 
vival function is given by 

P{W X > w) 


B(n - k + 1, k - r) 


k-r— 1 


S^rr 1 )- 


1 


1 


k + i + 1 {1 + (n — k + i + 1)1} 


(b) Use W x to obtain a 100a% prediction interval for Y k:n . 

(Lawless, 1971) 

(c) Another pivotal quantity is W 2 = 0* : „ — Y r:n )/Y r . n . Give an ex- 
pression for the survival function of W 2 for the special case k = r + 1. 
Lingappaiah (1973) has obtained an expression for the pdf of W 2 for 
the general case. 

(d) Given a choice between W x and W 2 to construct a prediction interval 
for Y k n , which one would you choose? Why? 

31. (a) Find the probability levels of 0'- :n ,y n _ /+1;n ) for 1 < i <. 5 and 

n = 10, when the intervals are used as confidence intervals for (i) the 
population median and (ii) the 25th percentile. Assume the sample is 
from a continuous distribution. 

(b) Find the probability levels of the intervals considered in (a) when 
they are used as tolerance intervals for 80% of the population values. 

32. Assuming that we are picking a random sample from a continuous 
population, find the smallest sample size n for which the probability level 
of the interval ( Y i:n ,Y n:n ) is at least 95% when it is used as (i) a 
confidence interval for the population median and (ii) a tolerance inter- 
val for 50% of the population values. 

33. Assume that the population of interest is continuous and approximations 
are acceptable! 

(a) Determine the probability level of (Y i:n ,Yj. n ) when used as an 
interval estimate of F~Kp) when (i) p = 6 . 5 , n = 100, i = 40, ; = 60, 
(ii) p = 0.1, n = 100, i = 8, j = 18. 

(b) Suggest a 95% confidence interval for F~\p) in each of the two 
cases in (a). 
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34 . Let X v ...,X n be a random sample from an absolutely continuous cdf 
F. Let F n (x ) represent the empirical cdf of the sample. 

(a) Show that as a function of x, F n (x) is a nondecreasing, right-continu- 
ous function bounded by 0 and 1. 

(b) For a fixed x, determine the distribution of nF n (x). Show that F n (x) 
is an unbiased and consistent estimator of F(x). What is the asymp- 
totic distribution of F n (x)? Would your answer change if F were 
discrete? 

(c) Let F~ l (t) = sup{x: F n (x) < t) for 0 < / < 1, and F„ -1 (l) =X n n . 
This is the inverse empirical cdf or the sample quantile function. 
Discuss the properties of F~ l (t). 

35 . The following is a classic data set taken from Grubbs (1950). It consists of 
a sample of 15 values derived from the observations on the “vertical 
semi-diameters” of Venus made by Lieutenant Herndon in 1846. 

-1.40, -0.44, -0.30, -0.24, -0.22, -0.13, -0.05,0.06,0.10,0.18, 
0.20,0.39,0.48,0.43, 1.01. 

Do you consider the sample minimum to be unreasonably low? Assume 
that the sample is taken from a normal population. 

36 . In a comparison of strength of various plastic materials, one characteris- 
tic of interest is the percent elongation at break. Grubbs (1969) has 
reported the following sample consisting of 10 measurements of percent 
elongation at break made on material labeled No. 23 in an experiment. 

2.02, 2.22, 3.04, 3.23, 3.59, 3.73, 3.94, 4.05, 4.11, 4.13. 

(a) Draw the Q-Q plot assuming the sample is from a normal popula- 
tion. 

(b) Test whether the sample minimum is an outlier. 

(c) Delete the second smallest value from the data set and repeat (b). 

(d) Comment on the outcomes of the analyses in (b) and (c) above. 
(Grubbs performs a test to decide whether the lowest two observa- 
tions are outliers to conclude that both of them are.) 

37 . The following data taken from Grubbs (1971) represent the mileages at 
the time of failure of 19 vehicles: 162, 200, 271, 302, 393, 508, 539, 629, 
706, 777, 884, 1008, 1101, 1182, 1463, 1603, 1984, 2355, 2880. 

(a) Draw a Q-Q plot to check whether the data can be thought of as a 
random sample from a two-parameter exponential distribution. 
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(b) Assuming the data represent a random sample from a two-parameter 
exponential distribution, obtain 95% confidence intervals for 0„ the 
location parameter and 0 2 , the scale parameter. 

(c) Also obtain the MVUE of the probability that the mileage at 
breakdown of a randomly chosen vehicle from this population ex- 
ceeds 200 miles. 

38 . Let Y x ,...,Y n be n independent continuous random variables such that 
all but Y n have cdf F and the outlier has cdf F*. Let / and /* be the 
pdfs of F and F*, respectively. 

(a) Show that the joint pdf of the order statistics T, Y n . „ is given 
by 

h(yu- -,y n ) = (» - i) ! /( yi) • • • /( yJ £ • 

y x <y 2 < ••• <y n . 

(b) Using the ideas that led to (2.2.13), show that the cdf of Y i:n is given 
by 

whenever 1 < i < n. Also obtain expressions for P(Y i:n <, y) for 
i = 1 and i = n. Note that the binomial sum on the right is nothing 
but Fj.n^iy). This is an extension of the second relation in Exercise 
2.23 to the case of a single outlier. 

(David and Shu, 1978) 

(c) Find the pdf of Y i:n . 

39 . In Exercise 38, let us assume F is Exp(l) cdf and F* is Exp(S) for some 
5 > 0. 

(a) Show that the probability that Y i:n is the outlier is given by 

r(n)r(n-i + (l/8)) 
fl r(n + (l/5))r(n -i + 1) 

(b) Hence show that when 5 > 1, P(Y r n is the outlier) increases as i 
increases. 

(Kale and Sinha, 1971) 
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(c) Suggest a test procedure for testing the hypothesis that one of the n 
observations comes from a population with mean 8 > 1. 

40. Show that the estimator T given by (7.9.3) is an unbiased estimator of n 
when 8 = 0, that is, when there is no outlier. 

41. Let Y/^ denote the ;'th order statistic from a random sample of size n t 

from the two-parameter exponential distribution /(y; 0,, o) = 
o ~ ! exp(-(y - 9j)/o), y > 0,, a > 0, for / = 1, . . . , k. Let = n 

and Y l:n denote the smallest value in the entire data set consisting of all 
the n observations. Our goal is to test the null hypothesis H 0 : 0, = • • • 

= h- 

(a) Show that the likelihood ratio test reduces to a test procedure based 
on the statistic 

T= ^lMY^-Y Un )/(k- 1) 

.(r/£ - r. ( !\)/(« - *) ' 

(b) Show that when H 0 is true, T has an distribution. If 

the alternative hypothesis just says that H 0 is false, when would you 
reject H 0 ? 

(c) What changes would you make in (a) and (b) when we have Type II 
censored samples Y^,, 1 < j < r t , where r t < n, for / = 1 , . . . , k. 

(Sukhatme, 1937; Khatri, 1974) 




CHAPTER 8 


Asymptotic Theory 


8.1. NEED AND HISTORY 

So far all our attention has been directed at the exact distribution theory for 
order statistics or some special functions of order statistics. An exception was 
when we considered approximations to the moments of order statistics. Our 
experience with the finite sample distribution theory shows time and again 
that the exact cdf is computationally messy except for some very special 
cases. This is true even when the sample size is small. Thus, one would 
naturally be interested in the large-sample behavior with the hope of finding 
some simple cdfs as good approximations to the actual cdf of the statistic of 
interest. 

There are several convergence concepts associated with the limiting behav- 
ior of a sequence of random variables. Convergence in distribution or weak 
convergence, convergence in probability, and almost sure convergence are the 
prominent ones. In the case of X n , the sample mean, these concepts lead us 
to the classical central limit theorem, weak law of large numbers, and strong 
law of large numbers, respectively. In this elementary introduction we will be 
mostly concerned with the weak convergence results for the order statistics. 
Even then, we present only the basic results. In the context of weak 
convergence, we are interested in identifying possible nondegenerate limit 
distributions for appropriately normalized sequences of random variables of 
interest. These limiting distributions can be of direct use in suggesting 
inference procedures when the sample size is large. 

As usual, we begin with a random sample X x ,...,X n from a population 
whose cdf is F. The asymptotic behavior of a single order statistic X i n 
depends on how i relates to n in addition to F. For example, the large-sam- 
ple behavior of the sample maximum is much different than that of either the 
sample minimum or the sample median. We will demonstrate this first for 
the standard exponential cdf in Section 8.2. In order to present general 
results in a systematic manner, we consider separately three distinct, non- 
exhaustive situations where X in is classified as one of the following: 
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(i) extreme order statistic when either i or n — i is fixed and the sample size 
n -* oo, (ii) central order statistic when i — [ np ] + 1 , 0 <p < 1 , where [•] 
stands for the greatest integer function, and (iii) intermediate order statistic 
when both i and n - i approach infinity, but i/n -* 0 or 1. We treat the 
extreme case in Sections 8.3 and 8.4; in the former we consider the sample 
maximum and minimum, and in the latter we look at other extreme order 
statistics and their limiting joint distributions. Section 8.5 is mainly concerned 
with the joint behavior of central order statistics. Toward the end of that 
section we give a brief account on intermediate order statistics. The last 
section deals with the asymptotic distribution of linear functions of order 
statistics commonly known as L statistics. In Chapter 7 we have seen that 
these statistics can provide efficient and robust estimators of certain popula- 
tion parameters. 

One important message about extreme order statistics is that if the limit 
distribution exists, it is nonnormal and depends on F only through its tail 
behavior. Early references include the work of Frechet (1927), who identified 
one possible limit distribution for X n .„. Soon after, Fisher and Tippett (1928) 
showed that extreme limit laws can be only one of three types. For the 
normal population, Tippett (1925) had earlier investigated the exact cdf and 
moments of X„. n and W n , the range. His heroic work involved compli- 
cated numerical integration and simulation techniques. He obtained very 
precise tables for the cdf of X n n and for E(W n ) for several values of n. 
von Mises (1936) provided simple and useful sufficient conditions for the 
weak convergence of X„ , n to each of the three types of limit distributions. 
Gnedenko (1943) established a rigorous foundation of the extreme value 
theory when he provided necessary and sufficient conditions for the weak 
convergence of the sample extremes. His work was refined by de Haan 
(1970). Smirnov (1949) studied in depth the asymptotic behavior of X i: „ 
when i or (u - i + 1) is fixed. Another important contributor, mostly on the 
statistical aspects, is Gumbel, who worked on this topic during the middle 
part of this century. Much of his work is reported in Gumbel (1958). 

In contrast to extreme values, the asymptotic distribution of a central 
order statistic is normal under mild conditions. Stigler (1973a) notes that as 
far back as 1818, Laplace had shown that the sample median is asymptoti- 
cally normal! More than a century later Smirnov (1949) obtained all possible 
limit distributions for central order statistics. Initial work on intermediate 
order statistics was done by Chibisov (1964). Daniell (1920) pioneered the 
systematic study of L statistics. But his work went unnoticed for quite some 
time. Stigler (1973a) summarizes Daniell’s contribution, while providing a 
historical account of robust estimation. 

There are several excellent books which deal with the asymptotic proper- 
ties of order statistics and L statistics. Galambos (1987), Resnick (1987), and 
Leadbetter, Lindgren, and Rootz6n (1983) discuss various asymptotic results 
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for extreme order statistics. Castillo (1988) presents some statistical applica- 
tions of the extreme value theory. Shorack and Wellner (1986), and Serfling 
(1980) are good reference sources on central order statistics and on L 
statistics. The recent book by Reiss (1989) investigates various convergence 
concepts and rates of convergence associated with all order statistics. 


8.2. EXPONENTIAL ORDER STATISTICS 

Before discussing the possible limit distributions for order statistics from an 
arbitrary cdf, we first consider the asymptotic results for the exponential 
parent distribution. These results, while being easy to prove, give us an idea 
about the nature of the limit distributions. 

With no loss of generality we assume the scale parameter 0 = 1 and recall 
once again the ever useful representation for exponential order statistics 
given in Theorem 4.6.1. It expresses X r n as a linear function of i.i.d. Exp(l) 
random variables denoted by Z/s as follows: 

La rJ Z r , 1 < i < n, (8.2.1) 

r= 1 


where 


1 /(n - r + 1), 1 < r <i, 

0, r > i. 


Let us now begin with the lower extremes. From (8.2.1), it is evident that 
since X l:n = Z l /n, nX Vn = Z, and, hence, nX i:n Z, as n -* <». More 
generally, when i is held fixed, Xj. n = {( Z x /n ) + ■■■ +(Z,/(n - i + 1))} 
and, therefore, nX i n (Z t + Z 2 + +Z ( ). In other words, the limit 

distribution of nX i n is T(i, 1), for any fixed i. 

In the case of upper extremes, it is more convenient to deal with the cdf. 
For the sample maximum, 


P{X n -.n ~ log« <zx) = 


( {1 - exp[-(x + log n)]}", 

\ 0 , 


x > - log n 
x < — log n 


= (1 - e x /n ) n , x > -log n 

-» exp{ -exp( -x)} , - oo < x < oo. 


Also, the limit distribution of the ith maximum, X n _ i+i . n , 


is related to the 
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limit distribution of X n:n for any fixed i. We will discover the precise 
relationship in the general context in Section 8.4. 

Now suppose i -» °° and n - i -* ». This includes both the central and 
intermediate cases. From (8.2.1) it is clear that in this situation we have a 
sum of ( independent random variables, none of which is dominant. We can 
apply Liapunov’s form of the central limit theorem to the sum in (8.2.1) and 
conclude that the limit distribution of (X l:n - /x ( .„)/cr J n is standard normal. 
This is pursued in Exercise 1. 

Before developing the general theory, let us summarize the conclusions of 
our experience with the exponential parent distribution. First, the limit 
distribution of X i:n need not be normal, in the extreme cases at least. 
Second, the upper and lower extremes may have different limit distributions. 
Third, in the extreme case, the limit law depends on In the next three 
sections we will explore further these findings when F is an arbitrary 
(discrete or absolutely continuous) cdf. 


8-3. SAMPLE MAXIMUM AND MINIMUM 

We now present a detailed discussion of the possible nondegenerate limit 
distributions for X n:n and then briefly go over parallel results for X l:n . 
Technically, it would be enough to consider only the maximum, since the 
sample minimum from the cdf F has the same distribution as the negative of 
the sample maximum from the cdf F* where F*(x) = 1 - F(-x). 

Since F n n (x) = {F(x)} n , X n , n f * F _1 (l), which is the upper limit of the 
support of F. In order to hope for a nondegenerate limit distribution, we will 
have to appropriately normalize or standardize X n . „. In other words, we look 
at the sequence {(X„ :n - a n )/b„, n 7> 1) where a„ represents a shift in 
location and b n > 0 represents a change in scale. The cdf of the normalized 
X„.„ is F n (a„ + b„x). We will now ask the following questions: 

(i) Is it possible to find a n and b„> 0 such that F"(a„ + b„x) -» G(x) 
at all continuity points of a nondegenerate cdf G? 

(ii) What kind of cdf G can appear as the limiting cdf? 

(iii) How is G related to F; that is, given F can we identify G? 

(iv) What are appropriate choices for a n and b n in (i)? 

In order to answer these questions precisely and facilitate the ensuing 
discussion, we introduce two definitions. 


Definition 8.3.1 (domain of maximal attraction). A cdf F (discrete or 
absolutely continuous) is said to belong to the domain of maximal attraction 
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of a nondegenerate cdf G if there exist sequences {a n } and {b n > 0} such that 

lim F n {a n + b n x) = G(x) (8.3.1) 


at all continuity points of G(x). If (8.3.1) holds, we will write F e 3>(G). 

Let If be a random variable whose cdf is G. Then, asserting that 
F e 3>(G) is equivalent to saying ( X n n - a n )/b n W. 

Definition 8.3.2. Two cdfs F, and F 2 are said to be of the same type if 
there exist constants a 0 and b 0 > 0 such that F,(a 0 + b 0 x) = F 2 (x). 

If random variables and W 2 with respective cdfs F, and F 2 are linearly 
related, then F, and F 2 are of the same type. 

Now suppose ( X n n - a n )/b n W where G is the cdf of W. Let {<*„} and 
{/3„ > 0} be two sequences of real numbers such that /3 n /b n -* b 0 and 
(a„ - a n )/b n -» a 0 as n -» ». Then, (X n:n - a„)//3„ 4 (IV - a 0 )/b 0 , whose 
cdf is G(a 0 + b 0 x). That is, the limit distributions of (X n:n - a n )/b n and 
(X n :n - <*„)// 3 n are of the same type. Conversely, it can also be shown that if 
F"(a„ + b n x) -* G(x) and F"(a„ + /3 n x) -» G 0 (x), then there exist con- 
stants a 0 and b 0 > 0 such that /3 n /b n -» b 0 and (a„ - a n )/b n -> a 0 and 
G 0 (x ) = G(a 0 + b 0 x). [See de Haan (1976) or Galambos (1987, p. 63) for a 
formal proof.] 

The informal discussion just given justifies the following observations. 
First, the choice of norming constants is not unique. Second, a cdf F cannot 
be in the domain of maximal attraction of more than one type of cdf. Thus, 
we will now turn our attention to identification of the types of G’s that are 
eligible to be the limiting cdfs in (8.3.1). 

Fisher and Tippett (1928) identified the class of such cdfs by a clever 
argument. First let n = mr, where m and r are positive integers. When 
m -* oo, while F"(a„ + b n x) -* G(x), it follows from the above discussion 
that [F m (a mr + b mr x)] r -* [G(a° + 6“*)]'', for some constants and > 0. 
Thus, G must be such that 

G"{a° n + b° n x) = G(x) (8.3.2) 


for some constants a° n and b° > 0, for all x and « > 1. The cdfs satisfying 
(8.3.2) are called max-stable cdfs. The solution to (8.3.2) depends on whether 
6“ > 1, 6“ < 1, or b° = 1. Frechet (1927) identified the solution in the first 
case, and shortly thereafter Fisher and Tippett (1928) characterized all 
possible solutions to (8.3.2). The result is summarized below. 
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Possible Limiting Distributions for the Sample Maximum 

Theorem 83.1. If (8.3.1) holds, the limiting cdf G of an appropriately 
normalized sample maximum is one of the following types: 


G, (*;a) = | 

fo, 

{ exp{ -x “}, 

jc ^ 0 

jc > 0; a > 0 

(8.3.3) 

^2(*; a) = | 

( exp{ -(-*)“}, 

jc < 0; a > 0 

(8.3.4) 

[1. 

jc £ 0 

G 3 (x) = « 

:xp{-exp( -*)}, 

-00 < JC < 00. 

(8.3.5) 


The first two limiting cdfs above involve an additional parameter a which 
is related to the tail behavior of the parent cdf F. The three cdfs G„ G 2 , and 
G 3 have names attached to each of them. The first one is referred to as the 
Frechet type, G 2 is called the Weibull type, and G 3 is often referred to as the 
extreme value cdf. Note that the negative of a Weibull random variable with 
shape parameter a has cdf G 2 (x; a). 

The three types of cdfs given in Theorem 8.3.1 appear on the surface to be 
unrelated. However, they can be thought of as members of a single family of 
distributions. For that purpose let us introduce a cdf G(jc; 0), which has the 
following form on its support: 

G(x;0) = exp{-(l + jc0 -1 )~ tf }, l+jcfl'X); -» < B < ». 

For 0 > 0, G(jc; 0), and G,(jc; 0 ) are of the same type. When 0 < 0, G(jc; 0 ) 
and G 2 (x; - 0 ) are of the same type. As 0 -» ±°°, G(jc; 0 ) -» G 3 (jc). The cdf 
G(jc; 0) is known as the generalized extreme value cdf, or as the extreme value 
cdf in the von Mises form. 

We have now answered question (ii) posed in the beginning of this section 
by showing that if F £ 3>{G), then G is either G, or G 2 or G 3 . We will now 
answer question (i) by presenting a set of necessary and sufficient conditions 
on F such that F £ ^(G,), i = 1,2,3. While doing so, we will also answer 
question (iii). 

Theorem 83.2 (necessary and sufficient conditions for weak convergence). 

(i) F £ ^(G,) iff F~'(l) = +» and there exists a constant a > 0 such 
that 

lim 1 — 7777 **““(- -log(G, (*;«))) (8.3.6) 

/-*oo I — t yl ) 

for all jc > 0. Further, if (8.3.6) holds, G,(jc) = G,(jc; a), and we 
write F £ &{G x (x\ a)). 
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(ii) F e 3>{G 2 ) iff F ’(1) is finite and there exists a constant a > 0 such 
that for all x > 0, 


1 -F(F-'(1) -ex) 
s io + 1 -F(F~'(1) -e) = *“ ( = 


•log G 2 ( —x] a)). (8.3.7) 


If (8.3.7) holds, G 2 (x) = G 2 (x\ a), and we write F e @(G 2 (x\ a)). 

(iii) F e &(G 3 ) iff E(X\X > c) is finite for some c < F _1 (l), and for all 
real x. 


Iim 

f-F-'U) 


1 - F(t + xE(X- t\X> t)) 

i-f(0 


= exp(-*)( = -log G 3 (x)), 
(8.3.8) 


where X represents the population random variable having cdf F. 

The conditional mean E{X - t\X > t) which appears in (8.3.8) is known 
as mean residual life in the reliability literature. 

This result is adapted from Theorems 2.1.1— 2.1.3 and Theorem 2.4.1 in 
Galambos (1987). The necessary and sufficient conditions given in the above 
theorem are equivalent to the original conditions used by Gnedenko (1943) 
to prove the domain of attraction results. Note that when G = G„ the upper 
limit of the support of F should be infinite and when G = G 2 , it should 
necessarily be finite. Sometimes, this is helpful in eliminating either G, or G 2 
as the possible limit distribution. 

Theorem 8.3.2 answers question (i) completely by providing a set of 
necessary and sufficient conditions on F so that {(X n . n - a n )/b n ) can be 
made to converge to a nondegenerate limit distribution. We will now present 
an example of an F for which none of these conditions hold. 

Example 8.3.1. Let F(x) = 1 - (log jc) — ', x > e. Since F -1 (l) = +», 
F £ 3f(G 2 ). Also, since 


lim 
/-* 00 


1 - F(tx) 

i-no 


lim 

t -*00 


Iog(0 

log(tt) 


= 1, 


(8.3.6) fails to hold. Thus, F € 2f(G x ). Finally, 

E{X\X>c) = f(\ogxy 2 dx 

* r 
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is infinite for all c and, hence, from part (iii) of Theorem 8.3.2, F £ 
3(G 3 ). Thus, we conclude that there do not exist a„ and b„> 0 such that 
( X n:n - a n )/b„ has a nondegenerate limit distribution. 

The necessary and sufficient conditions for F e 3(G) as given in 
(8.3.6)-(8.3.8) are quite often difficult to verify. For that reason, we now 
present sufficient conditions due to von Mises (1936), which are easy to check 
and are not very restrictive. But they are applicable only for absolutely 
continuous parent distributions. 

Theorem 8.33 (von Mises’ sufficient conditions for weak convergence). 
Let F be an absolutely continuous cdf and let h(x) = /(*)/{ 1 - F(x)}. 

(i) If h(x) > 0 for large x and for some a > 0, 


\imxh(x)=a, (8.3.9) 


then F e 3(G^x\ a)). 

(ii) If F _1 (l) < 00 and for some a > 0, 


lim (F l (l) - x)h(x) = a, (8.3.10) 

then F e 3(G 2 (x ; a)). 

(iii) Suppose h(x) is nonzero and is differentiable for x close to F~\ 1) 
[or for large x if F _1 (l) = »]. Then, F e 3(G 3 ) if 


d 

lim — 
ax 



(8.3.11) 


A simple proof of this result is given in de Haan (1976). The function h(x) 
appearing in the foregoing is the failure rate function which we have already 
seen in earlier chapters. (See, for example, Chapter 2, Exercise 15.) It has a 
physical interpretation in the reliability context when F is the cdf of a 
life-length distribution. 

Finally, we answer our last question regarding possible choices for the 
norming constants, a n and b n . Determining their values is as important as 
claiming their existence. These constants, while are not unique, depend on 
the type of G. Convenient choices, in general, are indicated in the following 
result. 
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Theorem 8.3.4 (norming constants). We can choose a n ,b n > 0 so that 
F n (a n + b n x) -* G(x) as follows: 

(i) a n = 0, b n = /-'(I - n" 1 ) if G = G,. 

(ii) a„ = F^Q), b n = F~'( 1) - F ‘(l - n~ ') if G = G 2 . 

(iii) a„ = F-'(l - n-‘), b n = F(* - a„|Z > a„) or F '(l - (ne)" 1 ) - 
F-'(l - n _, )if G = G 3 . 

Further, if (8.3.11) holds, one can choose b n = [nf(a n )}~ ’. 

Before looking at some examples, we present parallel results for the 
sample minimum. Theorem 8.3.5 describes the class of limiting cdfs for the 
normalized minimum. Theorem 8.3.6 gives necessary and sufficient condi- 
tions for F to be in a domain of minimal attraction, and suggests some 
convenient choices for the norming constants. 


Asymptotic Distribution of the Sample Minimum 

Theorem 8.3.5. Suppose there exist constants a* and b* > 0, and a 
nondegenerate random variable W* such that ( X l :n - a*)/b* W*. Then, 

G*, the cdf of W* must be one of the following types: 

(i) G*(x; a) = 1 — G,(—x; a), 

(ii) G*(x;a ) = 1 - G 2 ( -*;«), 

(Hi) G*(x ) = 1 - G 3 (—x), 

where G,, G 2 , and G 3 are given by (8.3.3), (8.3.4), and (8.3.5), respectively. 
Note that G*(x; a) is the Weibull cdf with shape parameter a. 

Theorem 8 3.6 (conditions for convergence and norming constants). 

(i) F e 3>(G*(x;a )); that is, the cdf of W* is Gf(x; a), iff F -1 ( 0) = 
— oo and 


F(tx) 

i™. 7(77 <8 312) 


for all x > 0. One can choose a* to be 0 and b* to be the absolute 
value of F _1 (l/n). 
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(ii) F g 3f(G*(x; a)) iff F ‘(0) is finite and 

F(F~ 1 (0)+ex) 

«l™+ F(F~ l iO)+e) = *“’ 

for all x > 0. Here, one can choose a* = F _1 (0) and b* 
- F-‘(0). 


(8.3.13) 
F-\\/n ) 


(iii) F g 2?iG*) iff E(X\X <, c ) is finite for some c > F -1 (0) and 


lim 

/-*•-'( o) 


F(t + xE(t - X\X <z t)) 

fU) 


exp(x), (8.3.14) 


for all real x, where the random variable X has cdf F. If F g 
3(G*), convenient choices for the norming constants are a* = 
F~\\/n) and b* = Eia* - X\X < a*). 

Example 8.3.1 (continued). We have already seen that, when Fix) = 1 
- (log x) -1 , x 2: e, the sample maximum cannot be normalized so that the 
limiting distribution is nondegenerate. How about the sample minimum? For 
that purpose, let us verify that (8.3.13) holds by using L’Hospital’s rule. 


F(e + ex) xf(e + ex) 

lim -h T = lim ~T, 7" 

e-o+ F(e + e) t-o+ f(e + e) 

x(e + e){log(e + e)} 2 
= lim ■ , 2 

«-o+ ( e + ex){log(e + ex)} 

= x. 

Thus F g 3{G*ix\ 1)). Since the quantile function is given by F~ l (u) = 
exp{l/(l - «)}, 0 < « £ 1, a* = e and b* = exp{n/(/i - 1)} - e. 

Example 8.3.2 (Weibull distribution). Let F be a Weibull (a) cdf where 
a is the shape parameter, that is, 

F(x) = 1 - exp( -x“), x > 0, a > 0. 

Note that Fix) = G|(x; a), and its failure rate function is hix) = ax“ -1 . 
Further, the quantile function is F~ l iu) = {-log(l - u)) p , where /3 = 1/a. 

In the case of the sample maximum, we can apply part (iii) of Theorem 
8.3.3 to conclude that (8.3.11) holds. That is, F e 3>iG^). From part (iii) of 
Theorem 8.3.4 it follows that a n = (log nf and b n = {/i/(a n )) _1 = 
{(log n) p ~ l }/a. Note that the other two forms for b n are not this easy to 
evaluate even though any one of these three forms is acceptable. However, 
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when a = 1 or when F is a standard exponential cdf, all these three forms 
for b„ do coincide, their common value being 1. We conclude that, with 
/3 = 1/a, [a(X nn — (log n)^)/(log n) p ~ x ) W, whose cdf is G 3 . In Section 
8.2 this is what we observed for the special case of a = 1. 

For the sample minimum, note that 

P(n' /a X l:n > x) = {1 - F(x/n l/a )}" 

- exp( x > 0. 

Hence, with a* = 0 and b* = n~ l/a , (X l:n - a*)/b* has the cdf G*(x,a ) 
for every n (not just in the limit). Note that G*(x; a) is a min-stable cdf. (See 
Exercise 2.) 

Example 8.3.3 (Pareto distribution). Suppose F is a Pareto (0) cdf 
where 0 is the shape parameter so that 1 - F(x) = x~ e , x > 1, 0 > 0. For 
determining the weak convergence of the sample maximum, this cdf is an 
ideal candidate for verifying (8.3.6) since for x > 0, 

i Z3?l= x -e 

l-F(t) 

whenever t > max(l, 1/x). Thus, (8.3.6) holds with a = 0, which means 
F e <&{.G^x\ 0)). From Theorem 8.3.4, we obtain a n = 0 and b n = F -1 (l - 
n ~ x ) = n 1/e . In other words, the limiting cdf of n~ 1/e X n:n is G x (x\0). But 
we have already seen this result before. (Can you recall?) For the limit 
distribution of X x . n , on using L’Hospital’s rule, it follows that (8.3.13) holds 
with a = 1. Hence, F e 2?(G*(x; 1)) and a* = 1 and b* = (1 — n~ x )~ 0/9) 
- 1 . 

Example 8.3.4 (standard normal distribution). Let Fix) = d>(x), the 
standard normal cdf. Since F -1 (l) = +<», £ 2iG 2 \ On using L’Hospital’s 

rule, we obtain, for x > 0, 


lim 

t~*cc 


1 - <D(tc) 

l -®(0 


= lim 

x<p(tx) 

t 00 

<ip(0 

I 00 ’ 

X < 1, 

= 1, 

X = 1, 

U 

x> 1. 


This means (8.3.6) does not hold and, hence, g&iGJ. Now the only 
possibility is G 3 . The necessary and sufficient condition (8.3.8) for e 
•SKG 3 ) is hard to verify and, thus, we will try to check whether the von Mises 
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sufficient condition (8.3.11) holds. For that purpose, let us note that 


d ( 1 \ _ d (l-4>(x) 

dx\h(x)J dx\ <p(x ) 


x{l - ^(x)} 
<p(x ) 


(8.3.15) 


Use L’Hospital’s rule twice while taking the limit of the first term in (8.3.15) 
as x -* oo to conclude that (8.3.11) holds. Hence, O e SKG 3 ) and by the 
symmetry of the standard normal pdf, clearly $ e 2>(G*). 

What about the norming constants? Since the quantile function d> -1 («) 
does not have a closed form, a„ = <D -1 (1 - n~ l ) cannot be explicitly exhib- 
ited, even though it can be computed for any given n. As suggested in several 
references, one may take 


1 log(4ir log n) 

2 ^2 logn 
and 

b n = 1/ y/21og n . 

These are not the only choices, nor are they the best ones. (Of course, we did 
not tell you what one means by “best.”) Hall (1979) has shown that the best 
rate of convergence of sup_ 00< ^ < Jd>"(a rt + b n x) - G 3 (x)| is achieved when 
a„ and b n are chosen such that n<p(a„) = a„ and b„ = a~ l . Let us now see 
how these relate to the choices suggested in Theorem 8.3.4. 

From (8.3.15) and the fact that (8.3.11) holds, we can infer that 1 - 
<f>(x) - <p(x)/x for large x. Thus, instead of using a„ = <b -1 (l - n~ l ) or 
d>(a„) = 1 - 1/n, one could use <p(a n )/a n = 1/n or, equivalently, n<p(a „ ) = 
a„. This was the choice of Fisher and Tippett (1928) also. Since the von Mises 
sufficient condition holds, one can take b n = [n<p(a n ))~\ which means b n - 
a~ l with this choice of a„. Hall showed that even with this optimal choice, 
the rate of convergence is rather slow (of the order of log n). 

Fisher and Tippett (1928) were the first to notice the slow rate of 
convergence of <f> n (a„ + b„x) to G 3 (x). They showed that the first four 
moments of the cdf <I>' , (a n + b n x) are closer to those of G 2 (x; a) for a 
suitably chosen a rather than to those of G 3 (x), even for n as large as 1000! 
This disparity between the so-called penultimate and ultimate limit behavior 
of the sample extremes in general has been the subject of some serious 
studies in recent years. 

Sample Extremes from Discrete Distributions 

All the examples we have discussed so far have dealt with absolutely 
continuous cdfs, even though our necessary and sufficient conditions in 
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Theorem 8.3.2 do not require F to be absolutely continuous. However, most 
of the well-known discrete F ’ s do not satisfy any of these conditions. Let us 
now see why. First, if F has only a finite number of points in its support, 
clearly X n:n will always take a finite fixed number of values. Thus the limit 
distribution would necessarily be degenerate. Distributions like the discrete 
uniform and binomial fall into this category. 

When F has an infinite number of points in its support, the following 
necessary condition for the validity of (8.3.1) will disqualify some of our 
favorite distributions. 

Theorem 8.3.7 (a necessary condition for weak convergence). If (8.3.1) 
holds and F is discrete, then as x -* F _1 (l), hix) -* 0 where hix) = fix)/ 
{1 - Fix - )}. 

A short proof of this theorem can be found in Galambos (1987, p. 84). For 
discrete distributions with nondecreasing failure rate (see Chapter 3, Exer- 
cise 18 for a definition), this implies that X n n cannot have a nondegenerate 
limit distribution. The geometric and the negative binomial distributions 
belong to this class. A version of Theorem 8.3.7 applicable to sample minima 
can be used to claim that even X l:n from these distributions do not have 
nondegenerate limit distributions. 

However, there do exist discrete distributions in the domain of attraction 
(maximal or minimal) of an extreme value distribution. Below is an example 
for the sample maximum. 

Example 8.3.5 (discrete Pareto distribution). Let Fix) = 1 - x 

>1, 8 > 0, where [•] represents the greatest integer function. Then for 
x > 0, 


l-Fitx) _( [r] 

1 -Fit) \[*]/’ 


if t > max( 1,1/x). 


Clearly, this tends to x~ B as t -* °°. Since (8.3.6) holds with a = 0, 
F e 9)iG x ix', 8)) as in the case of absolutely continuous Pareto distribution. 
The convenient norming constants are a n = 0 and b n = F _1 (l - n~ l ) = 
(n'/ fl + 1) = «'/* 


8.4. OTHER EXTREME ORDER STATISTICS 

We will now take up the question of possible nondegenerate limit distribu- 
tions for the top ith or the bottom ith order statistics. We will also look at 
the asymptotic joint behavior of the extreme i order statistics when i is held 
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fixed. Finally, we will study the asymptotic joint distribution of X i:n and 
X n . n . This will facilitate the job of finding the asymptotic distribution of the 
sample range W„ and the midrange V n . 

Let us begin by recalling an elementary result in calculus, which turns out 
to be extremely useful in our discussion. 

Lemma 8.4.1. Let { c n ,tt > 1} be a sequence of real numbers. Then, as 
n -* oo, c n -» c iff (1 - c„/n) n -» exp(-c). 

For a fixed x, let us define 

c„ * «{l - F(a n + b n x)}, (8.4.1) 

where a n and b n > 0 are the norming constants satisfying (8.3.1); that is, 
F n (a n + b n x) -» G(x). This can be expressed in terms of c n as (1 - c n /n) n 
-» exp(log G(x)). Then, from Lemma 8.4.1, it follows that for a given x, 
(8.3.1) holds iff c„ -» -log G(x). Thus, (8.3.1) is equivalent to saying that c n , 
given by (8.4.1), converges to -log G(x) at all continuity points of G(x) as 
n -* oo. Now consider 

P(X n _ i+l:n <a n + b n x) 

= E (”)( f K + b „x)} r {l -F(a n + b n x)}" ~ r 

r=n—i+\ 

= E (!)(c n /«)'(l ~c n /n) n ~ s , 

i = 0 

where c n is given in (8.4.1). Clearly, this is the cdf of a Bin(n, c n /n) random 
variable evaluated at O' - 1), which converges to the cdf of a Poisson 
(-log G(x)) random variable iff (8.3.1) holds. (Show it!) This simple argu- 
ment yields the following result. 

Theorem 8.4.1 (asymptotic distribution of an extreme order statistic). For 
any real x, F(X n:n <,a n + b n x) -» G(x) as n -» « iff for any fixed i > 1, 


P(X n -i + i:n^ a n + *>„*) “» E G(. x) { -. log G(x)) r /r !. (8.4.2) 

r - 0 

Thus, (8.3.1) holds, or F e 3{G) iff for some finite i, (8.4.2) holds for all x. 

The above theorem establishes the strong connection between the asymp- 
totic behavior of X n n and of X n _ i+Un . Note that the norming constants are 
the same and the function G(x) in (8.3.1) and (8.4.2) is the same. Thus, 
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(X n -i+u„ ~ a „)/b n can converge in distribution to a random variable whose 
cdf can be only one of the three types as given by (8.4.2) with G = G v G 2 , or 
G 3 . Theorem 8.3.2 provides necessary and sufficient conditions, and Theorem 
8.3.3 has sufficient conditions for this convergence. The norming constants a n 
and b„ to be used in (8.4.2) are provided in Theorem 8.3.4. For example, 
when F e 5KG 3 ), as in the case of a normal parent, 


lim F((X„_ i+l; „ 


- a n )/b n < x) = exp{ -exp( -x)} 


E 


r = 0 


exp( -rx) 


where a n = F -1 (l - n _I ) and b n = E(X - a n \X > a n ). 

The results for the rth order statistic are similar when i is fixed. When 
F e 3{G*), for any finite / > 1, 

P(X i:n >a* n + b* n x) -» E (1 - G*(x)}{-Iog[l - G*(x)]} r /r\. (8.4.3) 

r=0 


Asymptotic Joint Distribution of Extreme Order Statistics 

Now let us look at the joint asymptotic behavior of the top i (> 2) order 
statistics. To make our proofs simpler, let us assume F is absolutely continu- 
ous and that appropriate von Mises sufficient conditions of Theorem 8.3.3 
hold. It then follows that the pdf of (X n:n - a n )/b n converges to g(x), the 
pdf of G(x). [Even though this is quite intuitive, the proof is quite involved; 
see Resnick (1987, pp. 86) or Galambos (1987, pp. 156).] This means 

lim nbj(a„ + b n x)[F(a n + b n x)}"~ 1 = g(x), 

n -*oo 


which, in view of (8.3.1) implies 

lim nbj(a n + b n x) = (g(*)/G(x)}. (8.4.4) 


The joint pdf of (X n:n - a n )/b n ,...,(X n _ i+l:n - a n )/b n can be ex- 
pressed as 


{ p ( a n + b n x i)T ' n ( w - r + 1 )b„f(a n + b n x r ), x, > x 2 > • • ■ > x,.. 
r** 1 

From (8.3.1) and (8.4.4) we now can conclude that the above joint pdf 
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converges to 


So o(*i x i) - G(Xi)n[g(x r )/G(x r )}, x l >x 2 > ••• >x r 

(8.4.5) 

Similarly, in the case of lower extremes, the asymptotic joint pdf of 
(*i:„ ~ (X i n - a*)/b* is given by 

it 1 o(*i x,) = {1 - G*(^)}riU*(^)/[l ~ G*K)]}, 

r— 1 

x i < x i < " ' < *,• (8.4.6) 

Let us summarize these facts in the form of a theorem, which holds even 
when F fails to satisfy the von Mises conditions. 

Theorem 8.4.2. If ( X n:n - a n )/b„ has a nondegenerate limit distribution 
with cdf G as n -* «, then the joint pdf of the limit distribution of 

(X n n - a n )/b n (X n _ i+l:n - a n )/b„ is given by (8.4.5). Similarly, if 

(X { . „ - a*)/b* has a nondegenerate limit distribution with cdf G*, then the 

joint pdf of the limit distribution of (2f 1:il - a*)/b* (X,. n - a*)/b* is 

given by (8.4.6). 

We can obtain the limiting marginal pdf of the fth upper order sta- 
tistic either from the joint pdf given by (8.4.5) or from its limiting cdf 
given by (8.4.2). Either way we obtain the pdf of the limit distribution of 
(X n - i+l :n -a n )/b n as 


, , {-io g G(x i )r i / ^ 

8o)( x >) (•—[)! *(*')• 

Similarly, for (X i:n - a*)/b*, the limiting pdf is 


S*»( x i) = 


{ ~ log [ 1 - G*(* ,)]} 


i — 1 


■8*( x i)- 


(8.4.7) 


(8.4.8) 


Later on, in (9.3.6), we see that g* i} given in (8.4.8) is in fact the pdf of the 
(/ - l)th upper record value from the cdf G*. (See also Chapter 9, Exercise 
23.) The pdf g (j) , given in (8.4.7), is that of the (t - l)th lower record value 
from the cdf G. 
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Example 8.4.1 (Weibull distribution). In Example 8.3.2 we have seen 
that when F is a Weibull (a) cdf, F e 3(Gf) and F e 3(G*(x; a)). We 
have also determined the norming constants a n , b n , a*, and b*. Using (8.4.5) 
and (8.4.6), we obtain 


So =exp|- £* r J ■ exp(-(exp( -*,))), 

-00 < X: < 


S ( *i , )(*i, •••,*,) = exp(-oc?) 1!“*“ 0 <x, < 

r= 1 


<*!<», 
< X, < 00. 


Further, the marginal pdfs of the limit distributions of (A r n _ ( + 1:n - a n )/b n 
and (X in - a*)/b*, obtained, respectively, from (8.4.7) and (8.4.8), are 


and 


exp{-exp( -X;)} 

*(«)( *i) = exp(-a,.) ; , -oo <*, <oo, 


= t*x\ a 1 


exp( -xf ) 
('-!)! ’ 


Xj > 0 . 


When a = 1, g* 0 reduces to the pdf of a IXi, 1) random variable. This agrees 
with our conclusion in Section 8.2 that when F is Exp(l), the limit distribu- 
tion of nX i . n is IXi, 1). Also compare this with the conclusion drawn in 
(9.3.1). 

We will now turn our attention to the joint asymptotic behavior of the 
upper and lower extremes. To keep the details simple, let us look at just the 
sample maximum and the minimum. To avoid trivialities, assume that both 
these random variables have nondegenerate limit distributions. We then have 
the following result. 

Theorem 8.4.3 (asymptotic independence of sample maximum and mini- 
mum). If F e 3/(G) and F e 3(G*), then (X n . n - a n )/b n and (X Vn - 
a* )/b* are asymptotically independent random variables. 


Proof. Since F e 3(G*) we know that {1 - F(a* + b*y)} n -* 1 - 
G*(y) for all real y. If y < G*~ l ( 1), 1 - G*(y) > 0, and for such y’s, 
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{1 - F(a* + b*y)) -* 1 as n -» <». Hence, for large n and y < G* '(1), we 
can write 


P{{X n , n ~ a„)/b„ Z x, (X 1:n - a*)/b* > y) 

= [F(a„ + b n x) - F(a * + b*y)} n 




(8.4.9) 


where c„, given by (8.4.1), converges to -log G(x) for all real x, as n -* ». 
Since {1 - F(a* + b*y) } -* 1, it follows from Lemma 8.4.1 that the second 
factor on the right-hand side of (8.4.9) tends to exp(-(-log G(x))) = G(x). 
As noted earlier, the first factor on the right-hand side in (8.4.9) tends to 
{1 - G*(y)}. Hence, we conclude 

P((*n,n - a n )/b n Z x, (X l:n - a* ) /b* > y) ^ G(*){l - G*(y)} 

(8.4.10) 


as n -* 00 for all real x and y < G* -1 (l). For y ^ G* _1 (l) it is obvious that 

(8.4.10) holds since the left-hand-side probability tends to 0. Thus, we have 
shown that (8.4.10) holds for all real x and y. This establishes the asymptotic 
independence of the two sample extremes. □ 

We apply Theorem 8.4.3 to find the asymptotic distribution of W n , the 
sample range, and V n , the sample midrange, assuming / is symmetric. Then 
if F G SKG,), it is also in 9>(G*). When, without loss of generality, we take 
/ to be symmetric around zero, we can choose a* = -a n and b* = b n . Let 
(X n:n - a„)/b„ 4 W. Then, (X l:n + a n )/b n 4 W*, where W* l -IFand 
W and W* are independent. So we can conclude that (W n - 2 a n )/b n -» W 
- W* and V n /b n 4 (W + W*)/2 as n -> «. Note that the cdf of W - W* 
is that of the twofold convolution of the cdf G,. 

As an example let us choose F to be standard normal; that is, F = d>, in 
which case G = G 3 and G* = G*. Then W + W* has standard logistic 
distribution (see Exercise 12). But the pdf of W - W* does not have a closed 
form. For approximations to its pdf see David (1981, pp. 268-269). 

Even when F is asymmetric, Theorem 8.4.3 can be of use whenever 
b n /b* -» 1 as /!-*<». If b n /b* -> 0 or one of the extremes is the 
dominant contributor to the asymptotic behavior of W n or V n . For example, 
when F is an Exp(l)cdf, (X n n - log/j) has limiting cdf G 3 and nX l:n has 
cdf G 2 (x; 1). Thus X l:n 4 b and, hence, (W n - log n) also has limiting 
cdf G 3 . 
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8.5. CENTRAL AND INTERMEDIATE ORDER STATISTICS 

For 0 < p < 1, let i = [ np ] + 1, where [ np ] represents the integer part of np. 
Then, X t . n represents the pth sample quantile and is a central order statistic. 
When F is absolutely continuous with finite positive pdf at F~ '( p), X l n is 
asymptotically normal after suitable normalization. This basic result, which 
was introduced informally in Chapter 1, will be formally proved below. 


Theorem 8.5.1 (asymptotic distribution of a central order statistic). For 
0 < p < 1, let F be absolutely continuous with pdf / which is positive at 
F _1 (p) and is continuous at that point. For i = [np] + 1, as n -* °°, 


v^/(F- 1 (p)) ( ^ (i ^ ( ) P)) 4 N(0, 1). (8.5.1) 

Proof. As outlined in Chapter 1 itself, we first show that (8.5.1) holds 
when F is a Uniform (0, 1) cdf. Then we use the inverse probability integral 
transform X i n = F~ l (U i n ) to prove it for a general F satisfying the condi- 
tions stated in the theorem. 

To show that 


^(U i:n -p) ^N(0,p(l-p)), (8.5.2) 


as n and hence / approaches infinity, first we recall that U i:n is a 
Beta(i, n — i + 1) random variable. Thus, it can be expressed as 


Uf.n 


A „ 

A n + B n ’ 


(8.5.3) 


with A n = E \-\Z r and B n = H n r t- +x Z r , where Z/s are i.i.d. Exp(l) random 
variables. Since E(Z r ) = 1 and var(Z r ) = 1, from the central limit theorem it 
follows that {( A„ - /)/ fi } 4 7V(0, 1). Consequently, on recalling i = [np] + 
1, we obtain {( A„ -i)/{n) 4 A, where A is MO, p). Similarly, we can 
conclude that {( B n -(«-/+ 1))/ i/rT) 4 B, where B is MO, 1 - p). Since 
A n and B n are independent for all n, the limit random variables, A and B, 
are also independent. This means 


c n = ^{(l-p)(A n 


0 -p{B n ~ (n - i + 1))} 


converges in distribution to a MO, p(l - p)) random variable. Now, using 



224 


ASYMPTOTIC THEORY 


(8.5.3), we can write 


Jn(Vi:n ~P) = 


C„ ~ {(*' ~np~ 1)/V^} 

{{ A n + B n )/ n } 


(8.5.4) 


The numerator on the right-hand side in (8.5.4) converges in distribution to a 
MO, p(l - p)) random variable. By the weak law of large numbers, the 
denominator converges to 1 in probability. Hence, by Slutsky’s theorem, 
(8.5.2) holds. 

For an arbitrary cdf F, we use Taylor-series expansion for X i:n as done in 
Section 5.5, and the relation in (5.5.6), to write 


x,., +W:, -rt(/(F-‘(0,)))‘‘. 

where the random variable D n is between U { . „ and p. This can be rearranged 
as 


( 8-5 5 ) 

When / is continuous at F~\p), it follows that as «-»«>, f(F~ l (D n )) 4 
f(F~ l (p)). We use (8.5.2) and Slutsky’s theorem in (8.5.5) to conclude the 
proof of the asymptotic normality of A r [np]+1:n as given in (8.5.1). □ 


In our proof, we assumed that / is continuous at F~\p) to show that (8.5.1) 
holds. But the result is true even without that assumption. Further, note that 
it requires F to be differentiable and have positive pdf only at F~\p ); the 
cdf can have discontinuities at other points in its support. From (8.5.4), it 
follows that the asymptotic distribution of Jii (U, . „ — p) remains the same as 
long as Jn((i/n) - p] -* 0 as n -* ». 

Theorem 8.5.1 implies that the sample quantile X i: „ is a consistent 
estimator of F~\p). Further, (8.5.1) can be used to obtain an approximate 
confidence interval for F~\p) if either the form of / is completely specified 
around F~\p) or a good estimator of f(F~ l (p)) is available. We also know 
from Section 5.5 that = F~\p) and af. n - p(l - p)/{n[f(F~ l (p))] 2 } 
by the David-Johnson approximations. So X i n is an asymptotically unbiased, 
consistent, and asymptotically normal estimator_of F~\p). Now let us see 
how X„, the sample median, compares with X„, the sample mean, as an 
estimator of the population mean. 

Example 8.5.1. Suppose that p. and the population median (= F~\j)) 
coincide. Let us assume the variance a 2 is finite and f(p) is finite_and 
positive. For simplicity, let us take the sample size n to be odd. While X„ is 
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an unbiased, asymptotically normal estimator of p. with varCA',,) = <r 2 /n, X n 
is asymptotically unbiased and normal. If the population pdf is symmetric 
(around p), X n is also unbiased. Further, var(X n ) = {4 nlfip)] 2 ) -1 . Thus, as 
an estimator of p, the sample median would be more efficient than the 
sample mean, at least asymptotically, whenever [2 f(p)]~' < a. This condi- 
tion is satisfied, for example, for the Laplace distribution with pdf /( x\\l) 
= \ exp(-|jc - p\), -oo < x < oo. For this distribution, we know that X„ is 
the maximum-likelihood estimator of p (Chapter 7, Exercise 18), and that it 
is robust against outliers. Further, since f(p) = j, we can construct confi- 
dence intervals for p using the fact that \fn{X n - p) is asymptotically 
standard normal! 

Example 8.5.2 (Weibull distribution). For the Weibull (a) cdf, /( jc) = 
ajc“ -1 exp(— jc") and F~\u) = { — log(l - u)} p , where /3 = 1/a. Hence, 
from Theorem 8.5.1 we can conclude that 



( — log( 1 -p)f}±N 0, 


p{~ log( 1 ~ P )} 

a 2 (l -p) 


20 - 1 ) \ 


when / = [np] + 1, 0 < p < 1. When a = 1, or when F is Exp(l), the above 
simplifies to 


^{2f ( , n + log(l-p)}4N(o >T ^). 

In Section 8.2 we noted that when F is an exponential cdf, (X i:n - 
is asymptotically standard normal when i -» o° and n - i -* oo. 
When i = [np] + 1, as the above example illustrates, Theorem 8.5.1 can be 
used to make a nicer statement in the sense that it gives simple expressions 
for the norming constants. 

When the conditions of Theorem 8.5.1 do not hold, the asymptotic 
distribution of X i n may not be normal. As noted earlier, if Jn{{i/n ) - p) 
-* 0, 0 < p < 1 , X i n is asymptotically normal when F has positive pdf at 
F~\p). Smirnov (1949) has shown that if the conditions imposed on F do 
not hold, the limit distribution can be one of three other types, two of which 
are related to standard normal cdf. The remaining one is a discrete uniform 
cdf with two-point support. When the rate of convergence of (i/n) to p is 
slower, other distributions are possible. When i -* «, n - i -* oo such that 
i/n -* p, where 0 <, p < 1, Balkema and de Haan (1978) show that one can 
find parent cdfs F such that the limiting cdf of normalized X i n is any 
desired cdf! 
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Asymptotic Joint Normality of Central Order Statistics 

So far we have discussed in detail the asymptotic normality of a single central 
order statistic. This discussion extends in a natural manner to the asymptotic 
joint normality of a fixed number of central order statistics. This is made 
precise in the following result. 

Theorem 8.5.2. Let i r = [np r ] + 1, 1 <, r <, k, where 0 < p, < p 2 < 

<p k < 1. Assume that for each r, /(F _1 (p r )) is finite and positive. Then the 
joint distribution of {n(X i :n - F~ l (p r )\ 1 <, r < k, is asymptotically k- 
variate normal with zero mean vector and covariance matrix % A = (<r r f ), 
where <r A = p r (l ~ p s )/{f{F~\p r ))f{F~\p s ))}, \ <r <s <k. 

Evidently, even asymptotically, two central order statistics are dependent. 
This is in contrast with the asymptotic independence of lower and upper 
extremes which we noted in Theorem 8.4.3. Further the asymptotic covari- 
ance structure here is very close to that of uniform order statistics. 

The joint normality can be used to construct simultaneous confidence 
regions for two or more population quantiles. It is of great use in studying the 
asymptotic properties of estimators of location or scale parameters which are 
linear functions of a finite number of central order statistics. As discussed 
toward the end of Section 7.5, the patterned structure 2 A greatly simplifies 
the computations needed to produce asymptotically BLUEs of the location 
and scale parameters. 

Let us now look at an interesting application of Theorem 8.5.2. 

Example 8.5.3 (two-parameter uniform distribution). Let F be 
Uniform(p - a, p + a), 0 < p, < 0.5 and p 2 = 1 - p,. Then, with i = 
[np x ] + 1, j = [np 2 ] = n - i + 1, the quasirange W u .„ = X J:n - X i n and 
2 th midrange V i j n = (X j:n + X i:n )/2 are jointly asymptotically normal. 
Since F~\u ) = p - a + 2 ua, and /(F -1 («)) = (2<r) -1 , 0 < u < 1, the pa- 
rameters of the limit distribution are easy to compute. On applying Theorem 
8.5.2 we can conclude Vn^(W^, ;:n - 2(1 - 2p,)<r) -^M0,8pj(l - 2p,)<r 2 ) 
and 'IniVi J:n - p) MO, 2p,<r 2 ). Further, they are asymptotically inde- 
pendent! One can use V t , ;:n as an estimator of p and {H^ ;:n /2(1 - 2/7,)} as 
an estimator of a when we have the symmetrically doubly (Type II) censored 
sample, X i:n X n _ i+l:n . Compare them with the BLUEs of the parame- 

ters computed in Chapter 7, Exercise 26. (Note that our notations are slightly 
different here.) 

Asymptotic Normality of Intermediate Order Statistics 

We conclude this section with a brief discussion of the asymptotic behavior of 
intermediate order statistics. Let us call X n _ i+l:n an intermediate upper 
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order statistic if / -» » but i/n -» 0 as n -» ». The limit distribution of 
X„-j+i:„ depends on the rate of growth of i, and it could be either normal or 
nonnormal. We state an interesting result due to Falk (1989) which estab- 
lishes the asymptotic normality of X„_ l+1: „. His article is a good source for 
references on other limit results for intermediate order statistics. 


Theorem 8.53. Let F be an absolutely continuous cdf satisfying one of 
the von Mises conditions given in (8.3.9)— (8.3.1 1). Suppose i -» » and i/n -» 
0 as n -» oo. Then there exist norming constants a n and b n > 0 such that 
(X„-i+i :n ~ o„)/b n MO, 1). One can choose a n = F~'(l - i/n) and b n = 
41 /{nf (a „)}. A similar result holds for X t . n when i -> oo and i/n -» 0. 


As we know, most of the standard absolutely continuous distributions 
satisfy the von Mises conditions. One such distribution is the Weibull 
distribution. 


Example 8.5.4 (Weibull distribution). We have already seen the asymp- 
totic behavior of extreme and central order statistics from Weibull (a) cdf in 
Examples 8.3.2 and 8.5.2, respectively. This cdf satisfies (8.3.11) and has 
F~ l (u) = { — log(l — u)} fi with 0 = 1/a. On applying Theorem 8.5.3, it 
follows that, with a n = {log (n/i)} p and b n = 4l{[\o^n/i)] p ~ 1 )/a, (X n _ i+i n 
- a n )/b n is asymptotically standard normal, if i -» oo and i/n -» 0, as 
n -» oo. This implies that when a = 1 or when F is Exp(l), [X n _ i+1 . n - 
log(n/i')}/ 4i is asymptotically standard normal. 


8.6. LINEAR FUNCTIONS OF ORDER STATISTICS 

For several distributions, linear functions of order statistics provide good 
estimators of location and scale parameters. In Section 7.5 we saw two 
special members of this class when we derived the BLUEs of the location and 
scale parameters. We also noted in Section 7.9 that among such functions 
those which give zero or negligible weights to extreme order statistics have 
another desirable property of being robust against outliers and some devia- 
tions from the assumed form of F. We will now formally define linear 
functions of order statistics, quite often known as L statistics. 

Suppose a, „’s form a (double) sequence of constants. The statistic 


n 

L n 'll/ ®i, n : n 

(= 1 


( 8 . 6 . 1 ) 
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is called an L statistic. When used as an estimator, it is often referred to as 
an L estimator. Except when a i n = 0 for all / but one, the exact distribution 
of L„ is difficult to obtain in general. An exception is when F is an Exp(0) 
cdf. We know that, in that case, sample spacings do have a nice distribution. 
The other exceptions include the case where a,„ is free of n. In that case L„ 
is essentially the sample mean except for possibly a change of scale. Thus, the 
study of the asymptotic behavior of L„ becomes all the more important. 

A wide variety of limit distributions are possible for L n . For example, 
when a, „ is zero for all but one i, 1 ^ i £ n, L n is a function of a single 
order statistic. We have seen in previous sections the possible limit distribu- 
tions for X i:n which depend on how i is related to n. We have also seen that 
the limit distribution may not exist, at least for the extreme order statistics. If 
L„ is a function of a finite number of extreme order statistics or of a finite 
number of central order statistics, one can use their asymptotic joint distribu- 
tions (Theorems 8.4.2 and 8.5.2, respectively) to obtain the limit distribution 
of L n . When it depends only on a fixed number of central order statistics, the 
limit distribution is normal, under mild conditions on F. 

Even when the a, s are nonzero for many i’s, L n turns out to be 
asymptotically normal when the weights are reasonably smooth. In order to 
make this requirement more precise, let us suppose a,- „ in (8.6.1) is of the 
form J(i/(n + 1 ))/n, where J(u), 0 < u ^ 1, is the associated weight func- 
tion. In other words, we assume now that L„ can be expressed as 



The asymptotic normality of L„ has been established either by putting 
conditions on the weights or the weight function, or by assuming F to be 
close to an Exp(l) cdf. Serfling (1980, Section 8.2), and Shorack and Wellner 
(1986, pp. 664) discuss the asymptotic normality of L„ under various regular- 
ity conditions. We will present one result which assumes J to be sufficiently 
smooth with very little restriction on F. It is a simplified version of a result 
due to Stigler (1974) [see also Mason (1981)]. Its proof is beyond the scope of 
this book. 

First, let us define 


H(J,F) = f xJ(F(x))dF(x) 

J — oo 

= / V( a) ’(«)<*« 


(8.6.3) 
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and 


<r\J,F)~ iff J(F(x))J(F(y)){F(x)(l - F(y)} dxdy 

J J — OD<X<y <OD 

= 2 // •^(“ 1 )^(“ 2 )“ 1 (1 - u 2 ) dF -\ u x ) dF -\ u 2 ). 

J J 0 <u, <u 2 < 1 

(8.6.4) 


Theorem 8.6.1 (asymptotic properties of L„). Assume £|A'| 3 is finite 
where X represents the population random variable with cdf F. Let the 
weight function J(u) be bounded and be continuous at every discontinuity 
point of F -I (u). Further, suppose \J(u) - 7(y)| < /f|u - y| s+I/2 for some 
constant K and 5 > 0, 0 < u < y < 1, except perhaps for a finite number of 
values of u and v. Then, the following results hold: 


(i) lim Jn(E(L n ) - n(J,F)) = 0, 

n -*«> 

(ii) lim n var(L„) = a 2 (J, F), 

n —*oc 

(Hi) ^(L n -»(J,F))±N(0,* 2 (J,F)), 


where L n , ix(J, F), and a 2 U, F) are given by (8.6.2)— (8.6.4), respectively. 

The smoothness condition imposed on J above is known as a Lipschitz 
condition. We now apply this theorem to obtain the limit distributions of 
some L statistics. 

Example 8.6.1 (sample mean). When J(u) = 1, 0 < u < 1, L n =X n , 
pU, F) = p, and a 2 (J, F) = a 2 . Since J(u) is a constant, all conditions 
imposed on J _in Theorem 8.6.1 are satisfied. If E\X\ 3 is finite, we can 
conclude that X n is asymptotically normal. This is the central limit theorem 
for the sample mean. However, we know that the sample mean is asymptoti- 
cally normal even if the third moment of the cdf F does not exist. Further, 
note that we have E(L n ) = p (J, F) in this example. 

Example 8.6.2 (Gini’s mean difference). When X x and X 2 are i.i.d. with 
cdf F, E\X x - X 2 \ provides a measure of dispersion which we will denote by 
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9. Clearly, an unbiased estimator of 9 is Gini's mean difference, given by 


0 , = 


»(»-!) i-,; 


E E \X, ~ Xj\. 


(8.6.5) 


This may not appear to be an L statistic at the first glimpse, but it can be 
expressed as 


(” + 1 ) f 2 f 2i 

-l),., U + l 



( 8 . 6 . 6 ) 


In other words (n - l)G n /(n + 1) is an L statistic whose weight function, 
/(«) = 2(2 m - 1). This weight function is bounded and continuous. Hence, if 
E\X\ 3 < «, from Theorem 8.6.1, we can conclude that Vn(G„ - 9) 
MO, <t 2 (J, F)), where a 2 (J, F ) is computed from (8.6.4). 

When the X’s are Nip, a 2 ) random variables, it turns out that 9 = 
E\Xi - X 2 \ = 2a/ -fir . Thus, 'prrG n /2 is an unbiased L estimator of a, the 
standard deviation of a normal population. It is known to be highly efficient 
and is reasonably robust against outliers. Chemoff, Gastwirth, and Johns 
(1967) show that in the normal case, L n with Jiu) = <I> _i (m) provides an 
asymptotically efficient estimator of a. They also show that for the logistic 
distribution, the L statistic whose weight function is /(«) = 6 m(1 - u) pro- 
vides an asymptotically efficient estimator of the location parameter. 

Example 8.6.3 (discard-deviation). Let 0 < p < j and 


J(u) 


0<u<p, 

1 0, p<,u<,\-p, 

[(1/p), l-p<M^l. 


(8.6.7) 


The L n corresponding to /(«) in (8.6.7) is the difference between the 
averages of top k and bottom k order statistics where k - [np]. Daniell 
(1920) considered the asymptotic mean and variance of such a statistic for 
p = 0.25, and he called it discard-deviation. Hogg (1974) has suggested the 
ratio of such L„’s with different p’s as a measure of tail thickness of F. One 
can apply Theorem 8.6.1 to obtain the parameters of the limit distribution of 
L n whenever F~\u ) is continuous at p and 1 - p. 
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Asymptotic Distribution of the Trimmed Mean 

An important L statistic is the trimmed mean where either upper or lower or 
both extremes are deleted. Its weight function can be expressed as J(u) = 
( p 2 -p,) -1 , 0 < p, < u < p 2 < 1, and J(u) = 0, otherwise. If F~ l (u ) is 
continuous at p, and p 2 , one can apply Theorem 8.6.1. Stigler (1973b) has 
obtained a more general result exclusively for the trimmed mean. We state 
his result below; but first, let us begin with some notation. 

For 0<p,<p 2 <l, the L statistic given by 


S 


n 


l 

[np 2 \ - [np x \ 


l»Pl] 

E x i:n 


i = lnp |] + 1 


( 8 . 6 . 8 ) 


is called a trimmed mean, where the proportions p, and 1 - p 2 represent the 
proportion of the sample trimmed at either ends. Let 


a = F \P\) ~ F '(Pi-) and P = F X {Pt)- f '(/v) 


represent the magnitudes of jump of F 1 at the trimming proportions. 
Introduce a cdf H obtained by truncating F as follows: 


[0, xs.F~\p x ), 

H{x) = \{F{x) - p l )/{p 2 - p x ), F~\p x ) £x<F-\p 2 -), (8.6.9) 
(l, x > F~'(p 2 ). 


Finally, let and aft denote the mean and the variance of the cdf H, 
respectively. 

Theorem 8.6.2. Let 0 < p, < p 2 < 1 and n -» ». Then 4n (S n - p. H ) -4 
W where the limit random variable can be expressed as 

w = {y + [ F ~ l (P\) ~ Ph \ y i + [ F ~'(P 2 ) ~ Fh ] Y 2 ~ a max(0, T,) 

”2 "\ 

+/3 max(0, y 2 )}. (8.6.10) 

In this expression, the random variable Y is M0,(p 2 - p,)o^), the random 
vector (y„y 2 ) is bivariate normal with EiY,) = 0, var(>') = p i q h where 
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q, = 1 - p h i - 1,2, and cov(y,,y 2 ) = ~p 1 q 2 , and, further, Y and ( Y X ,Y 2 ) 
are mutually independent. 

Let us now examine the implications of Theorem 8.6.2. Whenever 0 < 
Pi < P 2 < 1> Ph an d a H are always finite; thus, the result is applicable in 
that case even when F does not have finite moments. To apply the theorem, 
we have to assume £{min(A", 0)} 2 is finite if p x - 0 and if p 2 = 1, we have to 
assume the existence of Efmax^, 0)} 2 . Further, when p x = 0, a = 0, and y, 
is degenerate at 0 and when p 2 = 1, /3 = 0, and Y 2 is degenerate at 0. In any 
case, we do not have to assume E\X\ 3 is finite, a condition necessary to apply 
Theorem 8.6.1. 

The representation (8.6.10) for the limit random variable W shows that 
Jn(S n - p H ) is asymptotically normal iff a = 0 and /3 = 0. That is, the 
weight function J(u) and the quantile function F~\u) do not have any 
common discontinuity points. Hence, Theorem 8.6.2 provides a necessary and 
sufficient condition for the asymptotic normality of the trimmed mean. Stigler 
(1973b) discusses how inference procedures based on S n are affected when 
its asymptotic distribution fails to be normal. 

We conclude this section as well as this chapter with two examples of 
special trimmed means. 

Example 8.6.4 (symmetric trimmed mean). Let p x - p < £ and p 2 = 
1 ~ P = Q. Then, S„ is known as p-trimmed mean. We have already encoun- 
tered S„ in Section 7.9 [recall (7.9.2)], where it was suggested as a robust 
estimator. As noted there, its distribution is less affected by the presence of a 
few outliers. Let us assume / is symmetric around p and that F -1 (u) is 
continuous at u = p (and hence at u = q). Then, in Theorem 8.6.2 we have 
p H = p and = {[f*-fybc 2 dF(x)]/(l - 2p)) - p 2 . Thus, Jn(S„ - p) 

W, where W is M0, </■£,). Further, = [cr^iX - 2p) + 2p(F -1 (p) - 
p) 2 ]/(l - 2p) 2 . 

When F is a Uniform(0, 1) cdf, F~\u) = u, p = and H is 
Uniform( p, q). Thus, crfi simplifies to (1 - 2p) z /12. Hence, for the standard 
uniform parent, yfn(S n - M0,(1 + 4p)/12), as «-»<». 

Example 8.6.5 (selection differential). Geneticists and breeders measure 
the effectiveness of a selection program by comparing the average of the 
selected group with the population average. This difference, expressed in 
standard deviation units, is known as the selection differential. Usually, the 
selected group consists of top or bottom order statistics. Without loss of 
generality let us assume the top k order statistics are selected. Then the 
selection differential is 


( 8 . 6 . 11 ) 
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where p. and <r are the population mean and standard deviation, respectively. 
Breeders quite often use E(D k n (p.,cr)) or D k „(p.,cr) as a measure of 
improvement due to selection. If Ac = n — [ tip ], then except for a change of 
location and scale, D k n (p.,a) is a trimmed mean with p x = p and p 2 = 1 . 
Further, the cdf H of (8.6.9) is obtained by truncating F from below at 
F~ i (p). 

We can apply Theorem 8.6.2 to conclude that y[n(D k n (p,,cr) - p. H ) 

(1 - p) -1 {7 + ( F~\p ) - ja H )y, - a max(0, 7,)}, where 7 is MO, qa ft) and 
Y k is N(0,pq), and, further, Y and 7, are independent. When F~ l (u ) is 
continuous at p, a (= F~ l (p) — F~ x {p - )) is zero and, consequently, 
yfn(D k „(p.,a) - p H ) 4 N(.0,q~ l {<r^ + p(F~ l (p) - p 2 H )}). In other words, 
as k -» oo, 


^N(0,crZ +p{F '(p) - p. 2 H }). (8.6.12) 


The selection differential has been used as a test statistic to test for 
outliers, and it has several desirable properties. Barnett and Lewis (1984) 
present simulated percentage points of D k n {p,a) for several n assuming F 
is standard normal. _ 

When (i is unknown, X n replaces it in (8.6.11) to yield the sample 
selection differential 


This is a linear function of order statistics even though it is not a trimmed 
mean. For this L statistic, the weight function is given by 


J(u) 


- \/<T, 0 < U < p, 

p/aq, p < u < 1. 


(8.6.13) 


If F _1 («) is continuous at p, D k n (X„,cr) is asymptotically normal. The 
norming constants can be obtained from Theorem 8.6.1. Let H (//*) be the 
cdf obtained by truncating F from below (above) at F~'(p). Then, using J 
given by (8.6.13) in (8.6.3) and (8.6.4), we obtain p(J, F) = (p H - p ,)/a and 




P\P a H f 




qv 
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respectively. Then, we can conclude that, as k -> », 

yfk(D k ' n (X n ,<T)-*-\v H -ri) N(0,qa 2 (J,F)), (8.6.15) 

where a 2 U, F ) is given by (8.6.14). The moments of F, and those of the 
truncated cdfs H and H*, are related. The relationships are 

P = QP-h + PPh* (8.6.16) 

and 


v 2 = + (ph ~ pf] + p{<th* + (Ph* ~ pf}- (8.6.17) 


The asymptotic variance_of i fkD kn (fi, a), given in (8.6.12), need not be 
less than that of }/kD k n (X n ,a), given in (8.6.15). See Exercise 27 for an 
illustration. 

If a is unknown, S, the sample standard deviation, is used in its place, 
which yields D k n (X n , S). But this statistic is no longer an L statistic. 
However, as noted in Chapter 4, Exercise 17, if we are_sampling from a 
normal population, the moments of D k n (X n , a) and D k n (X n , S ) are related. 
You may also recall that in Section 7.9 we have used D x J<X n , S ) for testing 
for a single (upper) outlier in a normal sample. For k > 1, Murphy (1951) 
proposed D k „(X n , S) as a test statistic for testing for k upper outliers. 

Further information about these selection differentials and the relevant 
references may be found in Nagaraja (1988b). 


EXERCISES 

1. (a) Check that Liapunov’s conditions hold while applying the central 

limit theorem for the X i n in (8.2.1) as i -» « and ( n — i ) -> °°. 

(b) Determine and a 2 n in (a), and find compact asymptotic approx- 
imations for them. 

2. (a) Paralleling the definition of max-stable cdf given in (8.3.2), define a 

min-stable cdf. (See Section 6.4.) 

(b) Show that G*, G*, and G* are all min-stable cdfs. As asserted in 
Section 6.4 these are the only distributions in the class of nondegen- 
erate min-stable cdfs. 
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3. For each of the following cdfs/pdfs determine whether (i) X„ . „ and (ii) 
X 1:n can be normalized so that the limiting distribution is nondegener- 
ate. If they can, determine the appropriate norming constants. 


(a) F( x) = x 6 ,0<x<,\,d>0 (Power-function distribution). 

(b) F(x) = 1/(1 + e~ x ), -oo < x < oo (Logistic). 

1 1 

(c) fix) = — • r, -oo < x < oo (Cauchy). 

i t 1 + jr 
-{x/t) a-l 


-, x > 0; a, 6 > 0 (Gamma(a, 0)). 


(d) fM - 

(e) Fix) = 1 - exp[-(x/0) 2 ], x > 0, 9 > 0 (Rayleigh). 

(f) Fix) = 1 - exp( -x/il - x)), 0 ^ x < 1. 

(g) Fix) = 1 + exp(l/x), x < 0. 

(h) Fix) = G,( x). 

(0 Fix) = G 2 ix). 

0) Fix) = G 3 ix). 


4. When F is a Poisson cdf, show that neither X n n nor X x . n can be 
normalized to yield a nondegenerate limit distribution (the relation 
between Poisson tail probability and Gamma cdf may turn out to be 
handy here!). 


5. When n -> oo, but i is held fixed, how are the asymptotic distributions of 
X i:n and X i:n related? Express the limiting cdf and pdf of X i:n in terms 
of those of X l n . 

6. Let iX n _ j+i:n - a n )/b n Wj, j = 1, . . . , k. Obtain explicit expressions 
for the joint pdf of iW v W k ) when G = G ( , i = 1, 2, 3. 


7. Let Z l ,...,Z k be i.i.d. Exp(l) random variables. Show that for the W ’ s 
defined in Exercise 6, the following representation holds: 


(H' 1 ,..^H' t )i(T 7 (Z 1 ),T 7 (Z 1 + Z 2 ),...,t 7 (Z 1 + ■■■ +ZJ), 



236 


ASYMPTOTIC THEORY 


where 


tj(x)=G *(exp( -x)) 


x~ l/a - 

V 

II 

O 

<-*-4 

-x t/a , 

if G = G 2 , 

-logx, 

if G = G 3 , 


8. (a) Using the representation for the Hy’s given in Exercise 7, or other- 

wise, show that when G = G 3 , 1V X - W 2 , W 2 - W 3 W k _ x ~ W k , 

and W k are independent random variables. 

(b) Determine the distributions of these spacings. 

(c) Using (a) and (b) above, or otherwise, find the mean and variance of 
W k when G = G 3 . These are the moments of ( k - l)th lower record 
value from G 3 . 

9. Let X x , X 2 , . . . be an infinite sequence of i.i.d. random variables with 

common cdf F. Let u x ,u 2 ,... be a sequence of nondecreasing real 
numbers. We say that an exceedance of the level u„ occurs at time / if 
Xj > u n . Let A n denote the number of exceedances of level u n by 
X x X„. ' 

(a) Determine the distribution of A 

(b) Suppose the s are chosen such that n{ 1 - F(u n )) -* A for some 
positive number A, as n -* ». Show that A n converges in distribution 
and determine its limit distribution. 

(c) Verify that the event {A r n _ i+) :n ^ u„] is the same as the event 
{ A n < k). Hence determine the limiting value of P(X n _ k+l n <, u n ), 
where k is held fixed and n -* <*>, if n{ 1 - F(u n )} -* A. 

(d) Compare your answer in (c) to the limiting distribution of the 
appropriately normalized X n _ k+l:n we obtained in Theorem 8.4.1. 

(Leadbetter, Lindgren, and Rootzen, 1983, Chapter 2) 

10 . Let A i and A 2 be two independent random variables where A t has cdf 
G 2 (x; 1) and A 2 has cdf G*(x; 1). Define B x = A x - A 2 and B 2 =A x + 
A 2 . 

(a) Determine the marginal pdfs of B x and B v 

(b) Are B x and B 2 independent? 

(c) Use (a) to obtain the limiting distribution of appropriately normal- 
ized sample range of a random sample from a Uniform(0, 1) popula- 
tion. Determine the associated norming constants. 

11 . In Example 7.4.2 we noted that for the Uniform(0, 0 + 1) distribu- 
tion, the MLE of 8 based on a random sample is of the form 8 = 
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c(Y„.„ - 1) + (1 - c)Y l:n , 0 < c < 1. Hence its asymptotic distribution 
depends on the choice of c. 

(a) Determine the asymptotic distribution of 8 for all c. 

(b) From Exercise 7.12, we know that when c = 0.5, 8 becomes the 
BLUE of 8. Use (a) to obtain the limit distribution of the BLUE. 
What are the norming constants? 

12. (a) Let A and B be two independent random variables where A has cdf 

G 3 and B has cdf G*. Obtain the cdf of the random variable 
X = A + B. 

(b) What is the pdf of the limiting distribution of the sample midrange 
from a standard normal population? 

13. Let F have pdf / such that fi8 + x) = fi8 - x ), for x > 0. 

(a) Show that X n , the sample median, is an unbiased estimator of 6 for 
both odd and even sample sizes. 

(b) Show that any L statistic with weight function Jiu ) is unbiased for 6 
if J(u) = /(l - u). 

(c) When F is a Cauchy cdf with location parameter 6, use X n to obtain 
an unbiased estimator of 8 and find the asymptotic variance of your 
estimator. 

14. Obtain the asymptotic distribution of the interquartile range (A' [3n/4]+ , 

- A r [n/4]+1:n ) under suitable assumptions. State all your assumptions. 

15. In Exercise 14, simplify your answer when F is a Nip, a 2 ) cdf. Use it to 
produce an asymptotically unbiased estimator of a which is a weighted 
interquartile range. 

16. Specialize Exercise 14 to the case where (i) F is Exp(0), (ii) F is the 
Pareto ( 8 ) cdf given in Example 8.3.3. 

17. Let i -» oo and i/n -* 0 as n -» <». Determine the asymptotic distribution 
of X n _ i+x . n and of X i n including the norming constants for the 
following cdfs: 

(a) F(x) = x 8 , 0 ^ x <, 1, 8 > 0. 

(b) Fix) — 1/(1 + e~ x ), — oo < x < oo. 

(c) Fix) = 1 - exp{-(x/0) 2 }, x > 0, 8 > 0. 



238 


ASYMPTOTIC THEORY 


18. Let X be an absolutely continuous random variable with cdf F and 
having variance a 2 . Show that 


-// 


— ®<jt<y <» 


F(x){l-F(y))dxdy. 


(Hint: Try integration by parts.) 

19. Show that the two expressions for Gini’s mean difference (G„) given by 

(8.6.S) and (8.6.6) are the same. 

20. (a) Determine E\X\ and £(max(0, A')), when X is MO, a 2 ). 

(b) Show that 'firG n /2 is an unbiased estimator of a assuming that we 
are sampling from a normal population. 

(c) Find E(.W) where W is the random variable corresponding to the 
limit distribution of the trimmed mean taken from a normal random 
sample. [Recall (8.6.10).] 

21. (a) When F is a standard uniform cdf, determine fi(J, F) and <r 2 (J, F) 

for the weight function J(u) = 2(2u - 1). 

(b) Now suppose F is Uniform(0, 0). Obtain an unbiased estimator of 0 
based on G„. Determine the parameters of the limit distribution of 
your estimator. 

22. Assume F is Uniform(0, 1) and 

J(u) = ll/(P-a), 0<a<u<fi<l 
' ; \0, otherwise. 

(a) Determine il(J, F) and <r 2 U, F) using the expressions (8.6.3) and 
(8.6.4). 

(b) Since the L statistic corresponding to this J(u ) is linearly related to a 
trimmed mean, one can use Theorem 8.6.2 to determine fi(J, F ) and 
a 2 U, F). Verify that the two answers match. 

23. In Theorem 8.6.2 obtain varOF) when F~\u) is continuous at p, andp 2 , 

0 <Pj <Pi < 1. 

24. Let /(«) be as given in (8.6.13) and F~ '(u) be continuous at p, 0 < p < 1. 

Show that 


p(/,F) = ( 
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and 


<t 2 (J,F) = -^{p^h + Wh* + [pPh + QPh* ~ F '(p)] 2 }. 

where q = 1 - p, and H and H* are the cdfs obtained by truncating F 
from below and from above at F~'(p), respectively. 

[Hint: Use Exercise 18 and integration by parts to evaluate cr 2 (J, F).] 

25. Assuming F, H, and H* are as in Exercise 24, verify (8.6.16) and 
(8.6.17). 

26. Let <I> be the standard normal cdf. Let the cdf H be obtained by 
truncating <I> from below at <F -1 (p), 0 < p < 1. Show that 

(a) p H = <p(<J> _1 (p))/(l - p), where <p is the standard normal pdf. 

(b) afi = <J>~\p)ii H + 1 - i u? H . 

(c) Use Exercise 24 and parts (a) and (b) and obtain expressions for p H * 
and a-"*. 

(d) For p = 0.95 and p = 0.99, determine p H , p H *, crfi, and 

27. (a) Simplify the expressions for the limiting variances of 4k D k n (j a, a) 

and 4k D k „(*„, cr) when F = < F. 

(b) Compare the two expressions. 

(c) Repeat (a) and (b) for the standard Laplace distribution whose pdf is 

given by /( x) = -» < x < °o. 

28. (a) Simplify p(J, F) and <r 2 (J, F ) when J(u) is given by (8.6.7) when the 

pdf of F is symmetric around zero and F _1 («) is continuous at p. 
Express these moments in terms of p H and a-fi where H is obtained 
by truncating F from below at F~\p). 

(b) Specialize the above expressions to the case (i) F = <t> and (ii) F is 
the standard Laplace cdf. 




CHAPTER 9 


Record Values 


9.1. THE HOTTEST JULY 4TH ON RECORD! 

As the temperature crept upward on July 4, 1957 in Riverside, California, 
many citizens monitored their radios to see whether, indeed, you could fry 
eggs on the sidewalk and whether, indeed, there had never been a July 4th 
like this in the weather department’s records. It was a record. Never once in 
the preceding 32 years for which data was available had the July 4th 
temperature exceeded the 108° registered on that day in 1957. A record for 
all times was the general feeling around town. But a little introspection 
suggested that eventually the record would be broken again. How long would 
it stand? What would be the new record, whenever it occurred? Questions 
like these prompted Chandler (1952) to formulate an appropriate body of 
theory dealing with record values in sequences of i.i.d. continuous random 
variables. It is quite remarkable that the genesis of record value theory can 
be pinpointed so exactly. Prior to 1952, even though people surely chattered 
about weather records and probably made and lost bar bets on them, they 
apparently neglected to develop a suitable stochastic model to study the 
phenomenon. Perhaps more remarkably, Chandler launched the study of 
record values but stayed aloof from further development of the area, save for 
cameo appearances as a discussant of the articles of other researchers. The 
21-year period following Chandler’s introduction of the topic saw a broad 
spectrum of researchers working in the record value vineyard. The major 
harvest was in by 1973 at which time, Resnick (1973) and Shorrock (1973) 
completed documentation of the asymptotic theory of records. Subsequently, 
there have been interesting new viewpoints introduced, relating record values 
with certain extreme processes. In addition, generalized record value se- 
quences involving nonidentically distributed observations have received more 
attention. Such models will be of interest in the case of improving popula- 
tions and provide partial explanations for the overabundance of record- 
breaking performances in athletic events. Our focus in the present chapter 
will be on the standard record value process just as it was introduced by 
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Chandler. The interested student may wish to refer to interesting and useful 
references surveying developments in the record value field. Glick (1978) 
using the intriguing title “Breaking records and breaking boards” provides a 
survey of the first 25 years. Contributions from the subsequent decade are 
discussed in Nagaraja (1988a). Incidentally, that 1957 temperature record still 
stands, though it was tied in 1989. 


9.2. DEFINITIONS AND PRELIMINARY RESULTS 
ON RECORD STATISTICS 

Let X v X 2 , . . . be a sequence of independent identically distributed random 
variables with common distribution function F. We will assume that F is 
continuous so that ties are not possible. There are a few results available on 
record values corresponding to discrete distributions (see, e.g., Exercises 
2-4), but the more elegant results are associated with the continuous case. 

An observation Xj will be called a record (more precisely an upper record ) 
if it exceeds in value all preceding observations, i.e., if Xj > X it Vi < j. 
Lower records are analogously defined. The sequence of record times {T n }” m „ 
is defined as follows. 


T 0 = 1, with probability 1, 

and for n > 1, 

T n = min{/: j > T n _ l , Xj > X T J. (9.2.1) 

The corresponding record value sequence {R,X-o * s defined by 

R„=X Ti n- 0,1,2 (9.2.2) 

Interrecord times, A„, are defined by 

K = T n -T n _ i; n — 1,2 (9.2.3) 

Finally, we introduce the record counting process 1 > where 

N n = {number of records among X x X n ) . (9.2.4) 

Monotone transformations of the 2f,’s will not affect the values of {T n }, {A„}, 
and { N n }. It is only the sequence [R„] that has a distribution which depends 
on the specific common continuous distribution of the X,-’s. For that reason it 
is wise to select a convenient common distribution for the X’s which will 
make the distributional computations simple. Thus, if {/?„}, {T n ), {A„}, and 
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[N n ] are the record statistics associated with a sequence of i.i.d. X’s and if 
y; = $(A' 1 ) (where <f> is increasing) has associated record statistics {/?'„}, {7^}, 
{A'J, and {Aft, then T n ± T n , A' n * A n> N' ± Aft and R' n ± #R n ). 


93. DISTRIBUTION OF THE nTH UPPER RECORD 

The friendliest common distribution for the A'/s is the standard exponential 
distribution. We will let {Z*} denote a sequence of i.i.d. Exp(l) random 
variables. Exploiting the lack of memory property of the exponential distribu- 
tion, it follows that the differences between successive upper records will 
again be i.i.d. standard exponential variables, i.e., {/?* - 7?* are i.i.d. 
Exp(l) [again the asterisk reminds us that the result is specific for an i.i.d. 
Exponential(l) sequence of observations]. From this it follows immediately 
that 


R* n ~ r(n + 1,1), n = 0,1,2,... . (9.3.1) 

From this we can obtain the distribution of R n corresponding to a sequence 
of i.i.d. X’s with common continuous distribution function F. Note that if X 
has cdf F, then — log[l - F(X)] ~ Exp(l), so that 

X = F~'( 1 -«-**), (9.3.2) 

where X* ~ Exp(l). Thus, X is a monotone increasing function of X* and, 
consequently, the nth record of the {*„} sequence, R n is related to the nth 
record R*, of the exponential sequence by 

R„ £ F-'( 1-e-*"*). (9.3.3) 

The following well-known expression for the survival function of T(n + 1,1) 
random variable 


P(R* >r*) =e-'* £ (r*) k /k\ (9.3.4) 

*=o 

allows an immediate derivation of the survival function of R„ as follows: 

P(R n > r) = P(F~'( 1 - e~ R *) > r) 

= P{ 1 -e~ R * > F(r)) 

= P(R* > -log(l ~F(r))) 

ft 

= [1 - nr)] L [— Iog(l - F(r))] k /k\. (9.3.5) 

*=o 
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If the distribution F is absolutely continuous with density /, then we may 
differentiate (9.3.5) and simplify the resulting expression to obtain 

/*„(>■) -/('•)[-l°*5(l - nO)]”/"!- (9-3-6) 

The joint density of the set of records R%,R* R* corresponding to an 

exponential sequence is easy to write down. From it, the joint density of 
records R 0 , R u ...,R n corresponding to any sequence of absolutely continu- 
ous XI s can be obtained by coordinatewise use of the transformation (9.3.3). 
The details are described in Exercise 23. 

Expressions (9.3.5) and (9.3.6) describe the distribution of the size of the 
nth record, when it occurs; they, of course, give no information about when 
we can expect to encounter the nth record. Such questions require study of 
the record times, interrecord times, etc., i.e., T n , A„, and N„. Before turning 
to a study of those variables, we may make a few comments on the approxi- 
mate distribution of R„ (the nth record) when n is large. Referring to (9.3.3) 
we see that 


i-O 


(9.3.7) 


where tj/iu) - F~K 1 - e ~ u ) and X*’s are i.i.d. Exponential(l) random vari- 
ables. Tata (1969) showed that a necessary and sufficient condition for the 
existence of normalizing constants a n and fi n > 0 so that (R n - a n )/(J n 
converges in distribution to a nondegenerate limit law is that there exists a 
nondecreasing function g(x) such that 


lim 


t- l (P„ x + a n ) -n 


8(x) 


(9.3.8) 


[where ^ is as defined following Eq. (9.3.7)], and in such a case the limiting 
distribution is given by 4>(g(x)), where 4> is the standard normal cdf (see 
Exercise 5). Later Resnick (1973) confirmed that there are only three forms 
that g(x) can take, namely; (i) g x (x) = x, -» <x <», (ii) g 2 (x) - a log x, 
x 2. 0, and (iii) g 3 (x) - -a log (-x), x £ 0 (where a is a positive constant). 
As is the case in the study of limiting distributions for maxima (Section 8.3), 
it is sometimes a nontrivial task to determine the appropriate normalizing 
sequences {«„} and {/3„} even when we know the appropriate form of the 
limiting distribution. 

As an example consider the record value sequence when the common 
distribution of the X t ’s is Weibull; i.e., F(x) - 1 - e~* r , x > 0, where y > 0. 



DISTRIBUTIONS OF OTHER RECORD STATISTICS 


245 


In this case 


<1 l ' l (v) = ~log[l - F(u)] = v y , 

and, consequently, if we choose a n = n l/y and /3„ = (n + 'Jn)' /y - n' /y 
and compute the limit in (9.2.8), we find g(x) = x and conclude in this case 
that R n is asymptotically normal. 


9.4. DISTRIBUTIONS OF OTHER RECORD STATISTICS 

Now we turn to the distributions of the T n ’ s, A„’s, and N n ’s. To get a flavor of 
the kinds of results obtainable and the nature of the arguments needed, 
consider first the distribution of T x , the time of occurrence of the first 
nontrivial record (recall T 0 = 1, since the first observation trivially exceeds all 
preceding observations). To get the distribution of T x , a simple argument is 
as follows. For any n > 1, 


1 

P(T X > n) = P(X x is largest among X x , X 2 , . . . , X n ) = -, (9.4.1) 

since any of the n A'/s is equally likely to be the largest. Thus, the distribu- 
tion of T x is given by 

n = 2 ' 3 (9A2> 

Observe that the median of T x is 2, but the mean is ». The last curiosity is 
the basis of Feller’s (1966) anecdote regarding the persistence of bad luck 
(the waiting time until someone has to wait longer than you in the super- 
market is infinite, so clearly you are singularly ill favored by the gods). This 
observation must be interpreted in the correct perspective. It refers to the 
unconditional mean for T x . Of course the conditional mean for T x , given a 
particular observed value of X x , is always finite (see Exercise 22). 

What about the joint distribution of several of the 7”s? A variety of 
conditional arguments can be used to resolve this issue [see, e.g., David and 
Barton (1962, pp. 181-183)]. A particularly appealing approach involves the 
introduction of record indicator random variables defined by 

/, = 1, w.p. 1 

and for n > 1, 


/„ = !, 
= 0 , 


if X„ > X x , . . X„_ x , 
otherwise. 


(9.4.3) 
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It turns out (see Exercise 6 ) that the 7„’s are independent random variables 
with 


/>(/„ = 1) = -, n = 1,2 (9.4.4) 

n 

The knowledge of the joint distribution of the 7„’s allow us to immediately 
write down expressions for the joint density of the record times. Thus, for 
integers 1 < n t < n 2 < • • • < n k , we have 

7 > (T, = n u T 2 = n 2 T k = n k ) 

= P(h = 0 7 n ,_, = 0, 7„, = 1, 7„ 1 + 1 = 0 I„ k = l) 

123 n, - 2 1 n t 1 

2 3 4 «[ - 1 «| «| + 1 n* 

= [(n, - l)(n 2 - 1),. ..,(n k - l)n*] -1 . (9.4.5) 

In principle we could use (9.4.5) to obtain the marginal distribution of T k , 
for k > 1. The resulting expression involves Stirling numbers of the first kind 
[see, e.g., Hamming (1973, pp. 160)]. To obtain this result, it is convenient to 
study the distribution of the record counting process [N„] defined in (9.2.4). 
Our record indicator variables {/„} are intimately related to [N„}. In fact 

N n =ilj. (9.4.6) 

y-i 

Recall that the 7/s are independent not identically distributed Bernoulli 
random variables. Immediately we may observe that 

Ew - i e ('i) - £ t 

y-i >-i J 

= log n + y, (9.4.7) 

where y is Euler’s constant. Analogously, 

" - 1 / 1 \ 
var (N n ) = E var(7 ; ) = E 7 1 “ 7 
/- 1 ; = 1 7 \ J I 



“ log n + y — — . 

0 


(9.4.8) 
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The above expressions for the mean and variance of N n justify the claim 
that records are rare. In fact, using Cantelli’s inequality (see Exercise 9) we 
can see that, for example, 


Wood 2: 20) < 0.036. (9.4.9) 


The probability generating function of N n (= E(s N ")) will be the product 
of the n generating functions of the Bernoulli random variables /,, / 2 , . . . , /„. 
Consequently (Exercise 10), we conclude that 


P(N n = k) = coefficient of s k in 


-7 FI (* + *- 1) 


nl t-i 


(9.4.10) 


The representation (9.4.6) allows us to immediately make certain observa- 
tions about the asymptotic behavior of N n . We have a strong law of large 
numbers : 


lim N n /\ogn = 1 a.s. (9.4.11) 

n-*o o 

(the corresponding weak law is readily dealt with; see Exercise 11). In 
addition, a central limit theorem holds: 


N n - log n 
0og n 


±N( 0 , 1 ) 


(9.4.12) 


(this is proved by verifying that the Liapunov condition is satisfied). 

Now we return to the problem of determining the marginal distribution of 
T k . Note the following relations among record related events. 

i T k >n} = { N n < k) 
and 

[T k -n] ={N n =k,N n _^k - 1} 

= = =*-!}• 


However, since the /„’s are independent random variables, it follows that the 
events {/„ = 1} and {N n _ l = k - 1) are independent. In addition, from 
(9.4.4) and (9.4.10), we know their corresponding probabilities. Consequently, 
we conclude that 


1 


n - 1 


P(T k = n) = — coefficient of s k 1 in FT (s + / - 1) 

n (n - 1)! ,^ =1 


= s k ~' 

J n- 1 


/«!, 


(9.4.13) 
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where the promised Stirling numbers of the first kind have made their 
appearance. 

Since T k > T, and E(T X ) = «, it follows that E(T k ) = °°, Vfc £ 1. In fact 
E(A k ) = °°, Vfc ^ 1; even the interrecord times have infinite expectation. To 
verify this observation consider 

oo 

E{A k \T k _ l = j) = 

/-I 

oo 

= E - 0, Ij +2 = o, . . . , 7 y+/ _, = 0, 7 y+/ = 1) 

/-i 

_ y l j j+1 > + l ~ 2 1 

i + 1 + 2 ;• + / - 1 ; + 7 

00 li 

— y* oo v/. 

(/+/)(/+/- 1) 

So unconditionally, £(A*) = oo. 

As k increases, grows rapidly. If instead we consider log T k we 
encounter better behavior. Renyi (1962) proved that 

log T k /k 1 (9.4.14) 

and 

{\ogT k -k)/Jk 4jV(0,l). (9.4.15) 


9.5. RECORD RANGE 

Consider a sequence of i.i.d. random variables {A r „}"_ 1 with common distri- 
bution function F. Now define the corresponding range sequence 




1 : n > 


n = 1,2,... . 


(9.5.1) 


Let {R„K_o d enote the sequence of record values in the sequence {V n }. The 
R„’s are the record ranges of the original sequence {X n }. A new record range 
occurs whenever a new upper or lower record is observed in the X H 
sequence. Note that R 0 = K, = 0. Generally speaking, the record range 
sequence is not a Markov chain. There is a closely related two-dimensional 
Markov chain, however. Let X' m and X" denote the current values of the 
lower record and upper record, respectively, in the X„ sequence when the 
m th record of any kind (upper or lower) is observed. Now {(A^,, AT")}* _ 0 
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does clearly constitute a Markov chain and we have 

R m =X" m -X' m (9.5.2) 

Now [following Houchens (1984)] a simple inductive argument allows one to 
determine the joint distribution of ( X ' m , Af" ) when the original X’s are 
Uniform(0, 1) random variables. Then, a change of variable leads eventually 
to the following expression for the density of the mth record range, R m , 
when the common distribution of the Af/s is F with corresponding density /: 

2 m 00 

fkS r ) = (m - 1 ) ! /-/^ + Z) ^ Z ^~ 1 ° g ^ 1 ~ F ( r + z > + dz - 

(9.5.3) 


In the simple case where the Af/s are Uniform(0, 1), (9.5.3) can be simplified 
to yield 


f*S') = 


2 m (l - r) 
( m - 1)! 


[ - log( 1 -r)]-\ 


0 < r < 1. (9.5.4) 


Since this is exactly the same distribution as the (m - l)st upper record 
corresponding to an i.i.d. sequence of random variables with common distri- 
bution F{x) = 1 - (1 - jc) 2 , further results on the distribution of R m can be 
gleaned from the material in Section 9.3. Why the mth record range of a 
uniform sample should have the same distribution as the (m - l)st upper 
record of a sequence of random variables each distributed as a minimum of 
two independent uniform (0, 1) variates, remains an enigma. 


9.6. BOUNDS ON MEAN RECORD VALUES 

Suppose that R n is the nth upper record corresponding to an i.i.d. sequence 
[X n ) with common continuous distribution function F(x). Assuming that a 
density fix) exists (though this is not technically needed for our final result), 
we can write, using (9.3.6), 

EiR n ) = f rfir)(- log[l - Fir)]) n /n\dr, (9.6.1) 

J — oo 


provided the integral exists. A change of variable u = Fir) allows us to 
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rewrite this in terms of the quantile function F~ l (u). Thus, 

E(R n ) = /V^H-logO - u)) n /n\du. (9.6.2) 

We recall that E\X^\ < °° was a sufficient condition to guarantee E\X n .„\ < » 
for every n. However, it is not a sufficient condition for the existence of 
E(R n ). To ensure that E(R„) exists for nonnegative X/s, it suffices that 
E(X i :(log A’,)") exists. A convenient general sufficient condition for the exis- 
tence of E(R n ) for every n is that F(|A r ,| p ) < » for some p > 1 (see 
Exercise 13). For our present purposes this is adequate, since we plan to 
assume that our AVs have finite means and variances. In fact, without loss of 
generality we will assume E(X t ) = 0 and varfA’,) and (= E(X 2 )) = 1. 

Now, mimicking the development in Section 5.4, we write, using (9.6.2) 
and /JF - *(«) du = 0, 

E(R n ) = f 1 F~ 1 (u)(g„(u) - A) du, (9.6.3) 

J o 


where 


8n(“) = [ — Iog(l - «)]"/«!• 

Applying the Cauchy-Schwarz inequality and recalling that fog„(u) du = 1 
and jo[F~\u)] 2 du = 1 , we get 

£(/?„) <;[S n -2A+A 2 ] 1/2 , (9.6.4) 

where 

= fgt(u) du 
J o 

- ( 2 ;)- (»■«) 

Now the right-hand side of (9.6.4) is smallest when A = 1, and so we 
eventually conclude that 


£(/?„) <^(2")- 1. (9.6.6) 

Equality can be achieved in (9.6.6). Let Z ( be such that P(Z, > z) = 
exp(-z 1/n ), z > 0. Then choose a n and b n so that X t = a„ + b n Z t has mean 
zero and variance 1. For such a translated Weibull sequence, equality obtains 
in (9.6.6) (Exercise 14). 
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More complicated improved bounds can be constructed if we impose 
further conditions on the distribution of the X/s (e.g., symmetry). See 
Nagaraja (1978) for details. 


9.7. RECORD VALUES IN DEPENDENT SEQUENCES 

A natural extension of the classical record value scenario is to postulate that 
the X/s, instead of being i.i.d., are identically distributed but not indepen- 
dent. The first case of interest involves an exchangeable sequence (Xj. By 
the famous de Finetti theorem, such a sequence is a mixture of i.i.d. 
sequences. Thus there exists a real random variable Z and a family of 
distribution functions (F z (x): :eR) such that, for each n, 

F Xl *„(*i ,■■■,*„) = /“Jfl^(^))^z(4- (9.7.1) 

Now the common distribution of the X/s is given by 

Fx(x) = f F z (x)dF z (z). (9.7.2) 

The distribution of the nth record, R n , can be computed by conditioning on 
Z and using the results of Section 9.3. Thus, from (9.3.5) and (9.3.6), 

P(K>r) = f P(R n >r\Z = z)dF z (z) 

J — oo 

= /" ([1-F z (r)] t [-log(l -F z {r))] k /k\\dF z {z) 

-°°\ *-o ) 

(9.7.3) 


and if densities f z (r ) exist, 

/*„(0 = f / z (0[-log(l -F z (r))] n /nldF z (z). (9.7.4) 

■' — oo 

Conditioning on Z allows us to compute the expected value of the nth record 
(assuming E\X x \ p < » for some p > 1 to guarantee that the integrals con- 
verge). Thus, 


E ( R n ) = / f F z 1 (“)[ — log(l - u)] n /n\dudF z (z) 

J — oo -Q 

= /‘ 1 F :_1 (m)[ — log(l - u)] n /n\du, 

J o 


(9.7.5) 
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where we have defined 

F~ l (u)= f F z ~ l (u)dF z (z). (9.7.6) 

J — 00 


Note that (9.7.5) is of the same form as is the expected nth record for an i.i.d. 
sequence. Note, however, that the function F~ l which appears in (9.7.5) is 
not in general the same as F* 1 , which would be the appropriate function if 
the A'-’s had been i.i.d. For example (see Exercise 19), the expected nth 
record for a certain exchangeable Pareto sequence coincides with the ex- 
pected nth record for an i.i.d. exponential sequence. The corresponding i.i.d. 
Pareto sequence has much larger expected records. It is not known to what 
extent this is a general result. The specific question at issue is whether, for a 
given marginal distribution F, the i.i.d. sequence has the maximal expected 
records among all exchangeable sequences. 

Record values have been studied in two other scenarios involving depen- 
dent identically distributed sequences. Biondini and Siddiqui (1975) studied 
the case where {XJ is assumed to be a stationary Markov chain. Only 
approximate results are obtained. The situation is not much more encourag- 
ing in the other case, which involves maximally dependent sequences [in the 
sense of Lai and Robbins (1978)]. For example, Arnold and Balakrishnan 
(1989, pp. 148-149) show that, in the case of a canonical maximally depen- 
dent sequence with common distribution function F, the expected first 
record is given by 


W=ir F-\ku)du, (9.7.7) 

which is larger than the corresponding expected first record corresponding to 
an i.i.d. sequence with common distribution F (see Exercise 20). Expressions 
for E(R n ) for n > 1 corresponding to maximally dependent sequences are 
not available. In all cases considered, record values for dependent sequences 
are still mysterious. 


9.8. RECORDS IN IMPROVING POPULATIONS 

Motivation for the study of improving populations was provided by the 
preponderance of new records encountered in athletic competitions such as 
the Olympic games. For i.i.d. sequences records are rare. In the Olympics, 
the records fall like flies. The simplest explanation involves an assump- 
tion that the X t sequence involved consists of independent variables which 
are stochastically ordered; i.e., X i ,^ st A r (+1 [which means P(Xj^x)^. 
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■P(A’ i+1 < Jt) for every *]. Yang (1975) proposed a model in which 

P(Y ( .<t) = [F 0 (x)]\ i = 1,2,... (9.8.1) 

for some distribution F 0 (x) and some sequence A, < A 2 < A 3 , . . . . In par- 
ticular he focused on the case in which A, = S'. Yang considered the 
interrecord time sequence A„ in such a setting and showed that A„ is 
asymptotically geometric. The distribution of the record values corresponding 
to the model (9.8.1) is nontrivial. We will illustrate by considering £(£,) 
when the underlying distribution F 0 (x ) is uniform, surely the simplest possi- 
ble case. First we evaluate P(R l > r ) conditioning on X l . Thus, for 0 < r < 1, 


£(£, > r) = P{X x > r) + f P(R l > r\X x = jc)A 1 jc a *~ i dx 
= P{X , >r) 

oo 

+ fZP(Xi>r)P(X 2 zx,...,X i _ l zx)\ lX *>-'dx 
J(\ ■ - 


'0 < = 2 


= 1 - r K ' + 52 (1 - rA ') / | I~I A, jc Ai 1 dx 


= 1 - r A| + £ (1 - r A -) r £ '- |A '. 

i = 2 = i A / 


Consequently, we have 

£(/?,) = fp(R l>r )dr 
J o 

— + E — 

1 Vi-l 


A. + 1 ,r 2 Ej:lA y \ L'r\\ j + 1 E' = 1 A, + 1 


(9.8.2) 


This expression does reduce to \ as it should when A ; = 1, V> [the i.i.d. 
Uniform(0, 1) case]. Computation of the expectation of other records in this 
setting is apparently difficult. 

Other improving population models have been studied in the literature. 
For example, Ballerini and Resnick (1985) consider models in which X l = 
y, + the y;’s are i.i.d. and is an increasing sequence. Another 
possibility permits the distribution of the X/s to change after each record has 
occurred [see Pfeifer (1982)]. Not surprisingly, none of the improving popula- 
tion models does a good job of modeling the Olympic games process. The 
obvious nonstationary nature of the Olympic process with its attendant 
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politics, boycotts, and the like undoubtedly precludes the development of a 
simple model. 


EXERCISES 

Refer to Section 9.2 for definitions. 

1. Verify that the record value sequence {i?X«o * s a Markov chain and 
identify the corresponding transition probabilities. Treat both discrete 
and continuous cases. 

2. If the *,.’s are i.i.d. geometric random variables, then R t - R 0 and R 0 
are independent. Verify this claim and determine whether this character- 
izes the geometric distribution among distributions with the nonnegative 
integers as support. 

3. If the Xj’s are i.i.d. geometric random variables, then R, - R 0 and R 0 
are identically distributed. Discuss this claim and its potential for charac- 
terizing the geometric distribution. 

4. Suppose that the A’/s are nonnegative integer valued and that E(R l - 
R 0 \R 0 ) = aR o + b. What can be said about the common distribution of 
the *,’s? 

(Korwar, 1984) 

5. Suppose that (9.3.8) holds where ip is as defined following (9.3.7). Verify 
that lim,,.,,,, P[(R n - a„)//3„ ^ *] = 4>(g(x)). 

6. Prove that the record indicator random variables {/X- 1 > defined in 
(9.4.3), are independent r.v.’s with P(I„ = 1) = 1/n, n = 1,2, ... . 

7. Show N u N 2 , ... is a Markov chain with nonstationary transition ma- 
trices. 

8. Show that {7X-i forms a stationary Markov chain with initial distribu- 
tion given by 


P(T t =;) = !/;(; - 1), j*2 
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and transition probabilities 


P(T n = *17^, =/)=//*(*-!), k>j. 


Hint: use the /■’ s. 

9. Verify that P(N l<m > 20) < 0.036. 

10. Explain why (9.4.10) is true. 

11. Verify that V„/log n 1 as n -> ». 

12. Verify that the density for the mth record range is as given as (9.5.3). 

13. (a) Verify that, assuming the X/s are nonnegative, E(R n ) exists if and 

only if E(X t (\og A',)") exists. 

(b) Verify that if E\xy < °° for some p > 1, then E(R n ) exists for 
every n. 

(Nagaraja, 1978) 

14. For a fixed value of n, determine the nature of the common distribution 
of the X/s for which equality obtains in (9.6.6). 

15. Let R' n denote the nth lower record corresponding to X v X 2 ,... i.i.d. 
Uniform(0, 1). Verify that R' n = n," =0 ^ where the V^’s are i.i.d. 
Uniform(0, 1). Use this to derive inductively the density of R'„. 

16. Suppose X l ,X 2 ,... are i.i.d. F with corresponding record value se- 
quence {)?„}“= o- Verify the following expressions for E(R n ): 

(i) [Uniform(a, b), F(x) = (x - a)/b, a < x < b] 

E(R n ) = a + (b- a)[l - 2^ n + l) ] 

(ii) [Weibull: F(x) = exp[ - (x/a) y \ , x > 0] 

E ( R n ) = ^r(n + i + ■y -1 )/r(n + i) 

(iii) [Pareto: F(x) = (x/a) “, x > a, a > l] 

£(/?„)= < 7(1 -«- 1 )- (,, + 1, . 
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17. Verify that the sequence of expected record values E(R n ), n = 0, 1, 2, . . . 
corresponding to {Xj i.i.d. F determines the distribution F. [Note that 
E(R n ) = /oF -, («X-log(l - u)]"/n\ du. Make the change of variable 
t = — log(l - «).] 

18. By keeping track of the fcth largest X yet seen, one may define a &th 
record value sequence [Grudzien and Szynal (1985)]. Most of the mate- 
rial in this chapter can be extended to cover such sequences. The key 
result is that the Arth record value sequence corresponding to the 
distribution F is identical in distribution to the (first) record value 
sequence corresponding to the distribution 1 - (1 - F) k . 

19. Assume that Z ~ r(2, 1) (i.e., / ? (z) = ze~ z , z > 0) and that, given 
Z = z, the X/s are conditionally independent with P{X i > x\Z = z) = 
e~ zx . Verify that this exchangeable sequence has the same sequence of 
expected record values as does an i.i.d. exponential sequence. 

20. Verify that the expected first record for a canonical maximally dependent 
sequence with common distribution F [given by (9.7.7)] is always larger 
than the expected first record corresponding to an i.i.d. sequence with 
common distribution F [given by (9.6.2) with n = 1]. 

21. If R„ has an IFR distribution, then so does R n + i (so if X x is IFR then 
all record values are IFR). 

(Kochar, 1990) 


22. Determine E(.T x \X x = x) in terms of the common distribution of the A/s 

( F x (x )). 


23. First verify that the joint density of R$, R’f , . . . , R* corresponding to an 
exponential sequence is given by 


& r * 


fill Rj( r 0 > r * > • • • > r * ) ~ e 

= 0 , 


o < r* < • • • <r* 
otherwise. 


From this obtain the following general expression for the joint density of 
records (/?„, . . . , R„) (assuming F is differentiable): 


m 


R„( r 0> r l>---> r n) 0 F( r n)) FI (i _ 


— oo < r 0 < r, < 


< r, < oo. 


Compare this with (8.4.6) which deals with extremes instead of records. 
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24. The following data represents the amount of annual (Jan. 1-Dec. 31) 
rainfall in inches recorded at Los Angeles Civic Center during the 
100-year period from 1890 until 1989: 


12.69, 

12.84, 

18.72, 

21.96, 

7.51, 

12.55, 

11.80, 

14.28, 

4.83, 

8.69, 

11.30, 

11.96, 

13.12, 

14.77, 

11.88, 

19.19, 

21.46, 

15.30, 

13.74, 

23.92, 

4.89, 

17.85, 

9.78, 

17.17, 

23.21, 

16.67, 

23.29, 

8.45, 

17.49, 

8.82, 

11.18, 

19.85, 

15.27, 

6.25, 

8.11, 

8.94, 

18.56, 

18.63, 

8.69, 

8.32, 

13.02, 

18.93, 

10.72, 

18.76, 

14.67, 

14.49, 

18.24, 

17.97, 

27.16, 

12.06, 

20.26, 

31.28, 

7.40, 

22.57, 

17.45, 

12.78, 

16.22, 

4.13, 

7.59, 

10.63, 

7.38, 

14.33, 

24.95, 

4.08, 

13.69, 

11.89, 

13.62, 

13.24, 

17.49, 

6.23, 

9.57, 

5.83, 

15.37, 

12.31, 

7.98, 

26.81, 

12.91, 

23.65, 

7.58, 

26.32, 

16.54, 

9.26, 

6.54, 

17.45, 

16.69, 

10.70, 

11.01, 

14.97, 

30.57, 

17.00, 

26.33, 

10.92, 

14.41, 

34.04, 

8.90, 

8.92, 

18.00, 

9.11, 

11.57, 

4.56 

(i) 

Obtain 

the observed values 

of the following 

sequences of 

record 


statistics: (a) T n 

, (b) A„, 

, (c) R 

„ (d) T' n 

, (e) A'„, 

(f) R' n . 




(ii) Do you think the rainfall is showing a decreasing trend? Suggest a 
test procedure based on a suitable record statistic. 


25. Suppose {/?„} is the record sequence corresponding to A'/s with continu- 
ous distribution function F. Suppose that {R n } is the record sequence 
corresponding to X’s with distribution function G. Suppose E(R n ) = 
E(R n ), n = 1,2,.... Prove that F = G. 


(Kirmani and Beg, 1984) 




Bibliography 


Abramowitz, M. and Stegun, I. A. (Eds.) (1965). Handbook of Mathematical 
Functions with Formulas, Graphs, and Mathematical Tables, Dover, New York. 
Ahsanullah, M. (1975). A characterization of the exponential distribution, in G. P. 
Patil, S. Kotz, and J. K. Ord, Eds., Statistical Distributions in Scientific Work (Vol. 
3, pp. 131-135), D. Reidel, Dordrecht, Holland. 

Ahsanullah, M. and Kabir, A. B. M. L. (1974). A characterization of the power 
function distribution, Canad. J. Statist. 2, 95-98. 

Aitken, A. C. (1935). On least squares and linear combinations of observations, Proc. 
Ry. Soc. Edinburgh 55, 42-48. 

Andrews, D. F., Bickel, P. J., Hampel, F. R., Huber, P. J., Rogers, W. H., and 
Tukey, J. W. (1972). Robust Estimates of Location, Princeton University Press, 
Princeton, NJ. 

Arnold, B. C. (1977). Recurrence relations between expectations of functions of 
order statistics, Scand. Actuar. J. 169-174. 

Arnold, B. C. and Balakrishnan, N. (1989). Relations, Bounds and Approximations 
for Order Statistics, Lecture Notes in Statistics No. 53, Springer- Verlag, New 
York. 

Balakrishnan, N. (1985). Order statistics from the half logistic distribution, J. 
Statist. Comput. Simul. 20, 287-309. 

Balakrishnan, N. (1986). Order statistics from discrete distributions, Commun. 
Statist. — Theor. Meth. 15(3), 657-675. 

Balakrishnan, N. (1989). A relation for the covariances of order statistics from n 
independent and non-identically distributed random variables, Statist. Hefte 30, 
141-146. 

Balakrishnan, N. (1990). Best linear unbiased estimates of the mean and standard 
deviation of normal distribution for complete and censored samples of sizes 
21(1)30(5)40, unpublished report, McMaster University. 

Balakrishnan, N. (Ed.) (1992). Handbook of the Logistic Distribution, Marcel 
Dekker, New York. 


259 



260 


BIBLIOGRAPHY 


Balakrishnan, N., Bendre, S. M., and Mauk, H. J. (1992). General relations and 
identities for order statistics from non-independent non-identical variables, Ann. 
Inst. Statist. Math. 44, 177-183. 

Balakrishnan, N. and Chan, P. S. (1992). Order statistics from extreme value 
distribution, I: Tables of means, variances and covariances, Commun. 
Statist. — Simul. Comput. (to appear). 

Balakrishnan, N., Chan, P. S., and Balasubramanian, K. (1992). A note on the 
best linear unbiased estimation for symmetric populations, submitted for publica- 
tion. 

Balakrishnan, N. and Cohen, A. C. (1991). Order Statistics and Inference: Estima- 
tion Methods. Academic, Boston. 

Balakrishnan, N. and Joshi, P. C. (1984). Product moments of order statistics from 
doubly truncated exponential distribution, Naval Res. Logist. Quart. 31, 27-31. 

Balakrishnan, N. and Kocherlakota, S. (1986). On the moments of order statis- 
tics from doubly truncated logistic distribution, J. Statist. Plann. Inf. 13, 117-129. 

Balakrishnan, N. and Malik, H. J. (1985). Some general identities involving order 
statistics, Commun. Statist. — Theor. Meth. 14(2), 333-339. 

Balakrishnan, N. and Mauk, H. J. (1986). A note on moments of order statistics, 
Am. Statist. 40, 147-148. 

Balakrishnan, N. and Malik, H. J. (1992). Means, variances and covariances of 
logistic order statistics for sample sizes 2 to 50, Selected Tables in Mathematical 
Statistics (to appear). 

Balakrishnan, N., Malik, H. J., and Ahmed, S. E. (1988). Recurrence relations and 
identities for moments of order statistics, II: Specific continuous distributions, 
Commun. Statist. — Theor. Meth. ( Statist . Rev.) 77(8), 2657-2694. 

Balasubramanian, K., Balakrishnan, N., and Mauk, H. J. (1992). Operator 
methods in order statistics, Aust. J. Statist, (to appear). 

Balkema, A. A. and de Haan, L. (1978). Limit distributions for order statistics I— II, 
Theor. Probab. Appl. 23, 77-92 and 341-358 (English version). 

Ballerini, R. and Resnick, S. I. (1985). Records from improving populations, 
J. Appl. Probab. 22, 487-502. 

Barlow, R. E. and Proschan, F. (1981). Statistical Theory of Reliability and Life 
Testing, To Begin With, Silver Spring, MD. 

Barnett, V. D. (1966). Order statistics estimators of the location of the Cauchy 
distribution, J. Am. Statist. Assoc. 61, 1205-1218. Correction 63, 383-385. 

Barnett, V. and Lewis, T. (1984). Outliers in Statistical Data, (2nd ed.), Wiley, 
New York. 

Bartholomew, D. J. (1963). The sampling distribution of an estimate arising in life 
testing, Technometrics 5, 361-374. 

Basu, D. (1955). On statistics independent of a complete sufficient statistic, Sankhya 
15, 377-380. 

Bennett, C. A. (1952). Asymptotic Properties of Ideal Linear Estimators, Ph.D. thesis, 
University of Michigan. 

Bernardo, J. M. (1976). Psi (digamma) function. Algorithm AS103, Appl. Statist. 
25, 315-317. 



BIBLIOGRAPHY 


261 


Beyer, W. H. (Ed.) (1991). CRC Standard Probability and Statistics Tables and 
Formulae, CRC Press, Boca Raton, FL. 

Bhattacharyya, G. K. (1985). The asymptotics of maximum likelihood and related 
estimators based on Type II censored data, J. Am. Statist. Assoc. 80, 398-404. 

Bickel, P. J. and Dorsum, K. A. (1977). Mathematical Statistics: Basic Ideas and 
Selected Topics, Holden Day, San Francisco. 

Biondini, R. and Siddiqui, M. M. (1975). Record values in Markov sequences, in 
M. L. Puri, Ed., Statistical Inference and Related Topics, (Vol. 2, pp. 291-352), 
Academic, New York. 

Birnbaum, A. and Dudman, J. (1963). Logistic order statistics, Ann. Math. Statist. 
34, 658-663. 

Bishop, Y. M. M., Fienberg, S. E., and Holland, P. W. (1975). Discrete Multivariate 
Analysis: Theory and Practice, MIT Press, Cambridge, MA. 

Blom, G. (1958). Statistical Estimates and Transformed Beta-Variables, Almqvist and 
Wiksell, Uppsala, Sweden. 

Blom, G. (1962). Nearly best linear estimates of location and scale parameters, in 
A. E. Sarhan and B. G. Greenberg, Eds., Contributions to Order Statistics (pp. 
34-46), Wiley, New York. 

Bondesson, L. (1976). When is the sample mean BLUE? Scand. J. Statist. 3, 
116-120. 

Borgan, O. (1984). Maximum likelihood estimation in parametric counting process 
models, with applications to censored failure time data, Scand. J. Statist. 11, 
1-16. Correction 11, 275. 

Bose, R. C. and Gupta, S. S. (1959). Moments of order statistics from a normal 
population, Biometrika 46, 433-440. 

Burr, I. W. (1955). Calculation of exact sampling distribution of ranges from a 
discrete population, Ann. Math. Statist. 26, 530-532. Correction 38, 280. 

Burrows, P. M. (1986). Extreme statistics from the Sine distribution, Am. Statist. 
40, 216-217. 

Cadwell, J. H. (1953). The distribution of quasi-ranges in samples from a normal 
population, Ann. Math. Statist. 24, 603-613. 

Casella, G. and Berger, R. L. (1990). Statistical Inference, Brooks/Cole, Pacific 
Grove, California. 

Castillo, E. (1988). Extreme Value Theory in Engineering, Academic, Boston. 

Chandler, K. N. (1952). The distribution and frequency of record values, J. Ry. 
Statist. Soc., Ser. B 14, 220-228. 

Chernoff, H., Gastwirth, J. L., and Johns, M. V., Jr. (1967). Asymptotic distribu- 
tion of linear combinations of functions of order statistics with application to 
estimation, Ann. Math. Statist. 38, 52-72. 

Chibisov, D. M. (1964). On limit distributions for order statistics, Theor. Probab. 
Appl. 9, 142-148. 

Clark, C. E. and Williams, G. T. (1958). Distributions of the members of an 
ordered sample, Ann. Math. Statist. 29, 862-870. 

Cohen, A. C. (1959). Simplified estimators for the normal distribution when samples 
are singly censored or truncated, Technometrics 1, 217-237. 



262 


BIBLIOGRAPHY 


Cohen, A. C. (1961). Tables for maximum likelihood estimates: Singly truncated and 
singly censored samples, Technometrics 3, 535-541. 

D’Agostino, R. B. (1986a). Graphical analysis, in R. B. D’Agostino and M. A. 
Stephens, Eds., Goodness-of-Fit Techniques (pp. 7-62), Marcel Dekker, New 
York. 

D’Agostino, R. B. (1986b). Tests for the normal distribution, in R. B. D’Agostino 
and M. A. Stephens, Eds., Goodness-of-Fit Techniques (pp. 367-419), Marcel 
Dekker, New York. 

Daniell, P. J. (1920). Observations weighted according to order, Am. J. Math. 42, 
222-236. 

David, F. N. and Barton, D. E. (1962). Combinatorial Chance, Griffin, London; 
Hafner, New York. 

David, F. N. and Johnson, N. L. (1954). Statistical treatment of censored data, I. 
Fundamental formulae, B'tometrika, 41, 228-240. 

David, H. A. (1970). Order Statistics, (1st ed.), Wiley, New York. 

David, H. A. (1981). Order Statistics, (2nd ed.), Wiley, New York. 

David, H. A. and Joshi, P. C. (1968). Recurrence relations between moments of 
order statistics for exchangeable variates, Ann. Math. Statist. 39, 272-274. 

David, H. A. and Shu, V. S. (1978). Robustness of location estimators in the 
presence of an outlier, in H. A. David, Ed., Contributions to Survey Sampling and 
Applied Statistics: Papers in Honour of H. O. Hartley (pp. 235-250), Academic, 
New York. 

Davis, C. S. and Stephens, M. A. (1977). The covariance matrix of normal order 
statistics, Commun. Statist. B6, 75-81. 

Davis, C. S. and Stephens, M. A. (1978). Approximating the covariance matrix of 
normal order statistics. Algorithm AS128, Appl. Statist. 27, 206-212. 

Davis, H. T. (1935). Tables of the Higher Mathematical Functions (Vols. 1 and 2), 
Principia Press, Bloomington. 

Davis, R. C. (1951). On minimum variance in nonregular estimation, Ann. Math. 
Statist. 22, 43-57. 

de Haan, L. (1970). On Regular Variation and Its Application to the Weak Conver- 
gence of Sample Extremes, Mathematical Centre Tract 32, Mathematics Centre, 
Amsterdam. 

de Haan, L. (1976). Sample extremes:.An elementary introduction, Statist. Neerland. 
30, 161-172. 

Desu, M. M. (1971). A characterization of the exponential distribution by order 
statistics, Ann. Math. Statist. 42, 837-838. 

Devroye, L. (1986). Non-uniform Random Variate Generation, Springer-Verlag, 
New York. 

Downton, F. (1966). Linear estimates with polynomial coefficients, Biometrika 53, 
129-141. 

Dudewicz, E. J. and Mishra, S. N. (1988). Modem Mathematical Statistics, Wiley, 
New York. 



BIBLIOGRAPHY 


263 


Epstein, B. (1960a). Statistical life tests acceptance procedures, Technometrics 2, 
435-446. 

Epstein, B. (1960b). Estimation from life test data, Technometrics 2, 447-454. 

Falk, M. (1989). A note on uniform asymptotic normality of intermediate order 
statistics, Ann. Inst. Statist. Math. 41, 19-29. 

Feller, W. (1966). An Introduction to Probability Theory and Its Applications (2nd 
ed.) (Vol. 2), Wiley, New York. 

Ferentinos, K. K. (1990). Shortest confidence intervals for families of distributions 
involving truncation parameters, Am. Statist. 44, 167-168. 

Ferguson, T. S. (1967). On characterizing distributions by properties of order 
statistics, Sankhya A 29, 265-278. 

Fisher, R. A. and Tippett, L. H. C. (1928). Limiting forms of the frequency 
distribution of the largest or smallest member of a sample, Proc. Cambridge 
Philos. Soc., 24, 180-190. 

Frechet, M. (1927). Sur la loi de probability de l’ecart maximum, Ann. Soc. 
Polonaise Math. 6, 92-116. 

Galambos, J. (1978, 1987). The Asymptotic Theory of Extreme Order Statistics, Wiley, 
New York (1st ed.). Kreiger, FL (2nd ed.). 

Galambos, J. and Kotz, S. (1978). Characterizations of Probability Distributions, 
Lecture Notes in Mathematics No. 675, Springer-Verlag, New York. 

George, E. O. and Rousseau, C. C. (1987). On the logistic midrange, Ann. Inst. 
Statist. Math. 39 627-635. 

Gibbons, J. D., Olkin, I., and Sobel, M. (1977). Selecting and Ordering Populations: 
A New Statistical Methodobgy, Wiley, New York. 

Glick, N. (1978). Breaking records and breaking boards, Am. Math. Monthly 85, 
2-26. 

Gnedenko, B. (1943). Sur la distribution limite du terme maximum d’une serie 
aleatoire, Ann. Math. 44, 423-453. 

Godwin, H. J. (1949). Some low moments of order statistics, Ann. Math. Statist. 20, 
279-285. 

Govindarajulu, Z. (1963a). On moments of order statistics and quasi-ranges from 
normal populations, Ann. Math. Statist. 34, 633-651. 

Govindarajulu, Z. (1963b). Relationships among moments of order statistics in 
samples from two related populations, Technometrics 5, 514-518. 

Govindarajulu, Z. (1968). Certain general properties of unbiased estimates of 
location and scale parameters based on ordered observations, SIAM J. Appl. 
Math. 16, 533-551. 

Graybill, F. A. (1983). Matrices with Applications in Statistics (2nd ed.). Wadsworth, 
Belmont, CA. 

Grubbs, F. E. (1950). Sample criteria for testing outlying observations, Ann. Math. 
Statist. 21, 27-58. 

Grubbs, F. E. (1969). Procedures for detecting outlying observations in samples, 
Technometrics 11, 1-21. 



264 


BIBLIOGRAPHY 


Grubbs, F. E. (1971). Approximate fiducial bounds on reliability for the two-parame- 
ter negative exponential distribution, Technometrics 13, 873-876. 

Grudzien, Z. and Szynal, D. (1985). On the expected values of &th record values 
and associated characterizations of distributions, in F. Konecny, J. Mogyorodi, 
and W. Wertz, Eds., Probability and Statistical Decision Theory (Vol. A, pp. 
119-127), Reidel, Dordrecht, Holland. 

Gumbel, E. J. (1954). The maxima of the mean largest value and of the range, Ann. 
Math. Statist. 25, 76-84. 

Gumbel, E. J. (1958). Statistics of Extremes, Columbia University Press, New York. 

Gupta, A. K. (1952). Estimation of the mean and standard deviation of a normal 
population from a censored sample, Biometrika 39, 260-273. 

Gupta, S. S. (1960). Order statistics from the gamma distribution, Technometrics 2, 
243-262. 

Gupta, S. S. (1962). Gamma distribution, in A. E. Sarhan and B. G. Greenberg, Eds., 
(pp. 431-450), Wiley, New York. 

Gupta, S. S. and Panchapakesan, S. (1974). On moments of order statistics from 
independent binomial populations, Ann. Inst. Statist. Math. Suppl. 8, 95-113. 

Gupta, S. S. and Panchapakesan, S. (1979). Multiple Decision Procedures: Theory 
and Methodology of Selecting and Ranking Populations, Wiley, New York. 

Gupta, S. S. and Shah, B. K. (1965). Exact moments and percentage points of the 
order statistics and the distribution of the range from the logistic distribution, 
Ann. Math. Statist. 36, 907-920. 

Hall, P. (1979). On the rate of convergence of normal extremes, /. Appl. Probab. 16, 
433-439. 

Halperin, M. (1952). Maximum likelihood estimation in truncated samples, Ann. 
Math. Statist. 23, 226-238. 

Hamming, R. W. (1973). Numerical Methods for Scientists and Engineers, McGraw- 
Hill, New York. 

Harter, H. L. (1961). Expected values of normal order statistics, Biometrika 48, 
151-165. Correction 48, 476. 

Harter, H. L. (1970). Order Statistics and Their Use in Testing and Estimation (Vol. 1 
and 2), U.S. Government Printing Office, Washington, D.C. 

Harter, H. L. (1978-1992). The Chronological Annotated Bibliography of Order 
Statistics, (Vols. 1-8), American Sciences Press, Columbus, Ohio. 

Hartley, H. O. and David, H. A. (1954). Universal bounds for mean range and 
extreme observation, Ann. Math. Statist. 25, 85-99. 

Hawkins, D. M. (1980). Identification of Outliers, Chapman and Hall, London. 

Hochberg, Y. and Tamhane, A. C. (1987). Multiple Comparison Procedures, Wiley, 
New York. 

Hoeffding, W. (1953). On the distribution of the expected values of the order 
statistics, Ann. Math. Statist. 24, 93-100. 

Hogg, R. V. (1974). Adaptive robust procedures: A partial review and some sugges- 
tions for future applications and theory (with comments), /. Am. Statist. Assoc. 
69, 909-923. 



BIBLIOGRAPHY 


265 


Hogg, R. V. and Craig, A. T. (1978). Introduction to Mathematical Statistics (4th 
ed.), MacMillan, New York. 

Horn, P. S. and Schlipf, J. S. (1986). Generating subsets of order statistics with 
applications to trimmed means and means of trimmings, J. Statist. Comput. 
Simul. 24, 83-97. 

Houchens, R. L. (1984). Record Value Theory and Inference, Ph.D. thesis, University 
of California, Riverside. 

Huang, J. S. (1975). A note on order statistics from Pareto distribution, Scand. 
Actuar. J., 187-190. 

Huang, J. S. (1989). Moment problem of order statistics: A review, Int. Statist. Rev. 
57, 59-66. 

Huber, P. J. (1981). Robust Statistics, Wiley, New York. 

Huzurbazar, V. S. (1976). Sufficient Statistics: Selected Contributions, Marcel Dekker, 
New York. 

Joshi, P. C. (1971). Recurrence relations for the mixed moments of order statistics, 
Ann. Math. Statist. 42, 1096-1098. 

Joshi, P. C. (1973). Two identities involving order statistics, Biometrika 60, 428-429. 

Joshi, P. C. (1978). Recurrence relations between moments of order statistics from 
exponential and truncated exponential distributions, Sankhya B 39, 362-371. 

Joshi, P. C. (1979a). A note on the moments of order statistics from doubly truncated 
exponential distribution, Ann. Inst. Statist. Math. 31, 321-324. 

Joshi, P. C. (1979b). On the moments of gamma order statistics, Naval Res. Logist. 
Quart. 26, 675-679. 

Joshi, P. C. (1982). A note on the mixed moments of order statistics from exponential 
and truncated exponential distributions, J. Statist. Plann. Inf. 6, 13-16. 

Joshi, P. C. and Balakrishnan, N. (1981). An identity for the moments of normal 
order statistics with applications, Scand. Actuar. J., 203-213. 

Joshi, P. C. and Balakrishnan, N. (1982). Recurrence relations and identities for 
the product moments of order statistics, Sankhya B 44, 39-49. 

Kagan, A. M., Linnik, Yu. V., and Rao, C. R. (1973). Characterization Problems in 
Mathematical Statistics, Wiley, New York (English translation). 

Kale, B. K. and Sinha, S. K. (1971). Estimation of expected life in the presence of an 
outlier observation, Technometrics 13, 755-759. 

Kaminsky, K. S. and Rhodin, L. S. (1985). Maximum likelihood prediction, Ann. 
Inst. Statist. Math. 37, 507-517. 

Kennedy, W. J. and Gentle, J. E. (1980). Statistical Computing, Marcel Dekker, 
New York. 

Khatri, C. G. (1974). On testing the equality of location parameters in k-censored 
exponential distributions, Aus. J. Statist. 16, 1-10. 

Kirmani, S. N. U. A. and Beg, M. I. (1984). On characterization of distributions by 
expected records, Sankhya A 46, 463-465. 

Kochar, S. C. (1990). Some partial ordering results on record values, Commun. 
Statist. — Theor. Meth. 19( 1), 299-306. 



266 


BIBLIOGRAPHY 


Korwar, R. M. (1984). On characterizing distributions for which the second record 
value has a linear regression on the first, Sankhya B 46, 108-109. 

Krishnaiah, P. R. and Rizvi, M. H. (1966). A note on recurrence relations between 
expected values of functions of order statistics, Ann. Math. Statist. 37, 733-734. 

Kuczma, M. (1968). Functional Equations in a Single Variable, PWN-Polish Scientific 
Publications, Warsaw. 

Lai, T. L. and Robbins, H. (1978). A class of dependent random variables and their 
maxima, Z. Wahrsch. Verw. Gebiete 42, 89-111. 

Laurent, A. G. (1963). Conditional distribution of order statistics and distribution of 
the reduced ith order statistic of the exponential model, Ann. Math. Statist. 35, 
1726-1737. 

Lawless, J. F. (1971). A prediction problem concerning samples from the exponen- 
tial distribution, with application in life testing, Technometrics 13, 725-730. 

Lawless, J. F. (1977). Prediction intervals for the two-parameter exponential distri- 
bution, Technometrics 17, 255-261. 

Lawless, J. F. (1982). Statistical Models & Methods for Lifetime Data, Wiley, 
New York. 

Leadbetter, M. R., Lindgren, G., and Rootzen, H. (1983). Extremes and Related 
Properties of Random Sequences and Processes, Springer-Verlag, New York. 

Lehmann, E. L. (1983). Theory of Point Estimation, Wiley, New York. 

Lieblein, J. (1955). On moments of order statistics from the WeibuII distribution, 
Ann. Math. Statist. 26, 330-333. 

Likes, J. (1974). Prediction of rth ordered observation for the two-parameter expo- 
nential distribution, Technometrics 16, 241-244. 

Lingappaiah, G. S. (1973). Prediction in exponential life testing, Can. J. Statist. 1, 
113-118. 

Lloyd, E. H. (1952). Least-squares estimation of location and scale parameters using 
order statistics. Biometrika 39, 88-95. 

Ludwig, O. (1959). Ungleichungen fur Extremwerte und andere Ranggrossen in 
Anwendung auf biometrische probleme, Biom. Z. 1, 203-209. 

Lurie, D. and Hartley, H. O. (1972). Machine-generation of order statistics for 
Monte Carlo computations, Am. Statist. 26, 26-27. 

Lurie, D. and Mason, R. L. (1973). Empirical investigation of several techniques for 
computer generation of order statistics, Commun. Statist. 2, 363-371. 

Malmquist, S. (1950). On a property of order statistics from a rectangular distribu- 
tion, Skand. Aktuar. 33, 214-222. 

Mason, D. M. (1981). Asymptotic normality of linear combinations of order statistics 
with a smooth score function, Ann. Statist. 9, 899-904. 

Mehrotra, K. G. and Nanda, P. (1974). Unbiased estimation of parameters by 
order statistics in the case of censored samples, Biometrika 61, 601-606. 

Mehrotra, K. G., Johnson, R. A., and Bhattacharyya, G. K. (1979). Exact Fisher 
information for censored samples and the extended hazard rate functions, 
Commun. Statist. — Theor. Meth. 8, 1493-1510. 



BIBLIOGRAPHY 


267 


Melnick, E. L. (1964). Moments of Ranked Poisson Variates, M.S. thesis, Virginia 
Polytechnic Institute. 

Melnick, E. L. (1980). Moments of ranked discrete variables with an application to 
independent Poisson variates, /. Statist. Comput. Simul. 12, 51-60. 

MINITAB (1990). Statistical Software Release 6.2. Minitab, State College, Pennsyl- 
vania. 

Mood, A. M., Graybill, F. A. and Boes, D. C. (1974). Introduction to the Theory of 
Statistics (3rd ed.), McGraw-Hill, New York. 

Moriguti, S. (1951). Extremal properties of extreme value distributions, Ann. Math. 
Statist. 22, 523-536. 

Moriguti, S. (1954). Bounds for second moments of the sample range, Rep. Stat. 
Appl. Res. JUSE 2, 99-103. 

Murphy, R. B. (1951). On Tests for Outlying Observations, Ph.D. thesis, Princeton 
University. 

Nagaraja, H. N. (1978). On the expected values of record values, Aus. J. Statist. 20, 
176-182. 

Nagaraja, H. N. (1979). Some relations between order statistics generated by 
different methods, Commun. Statist. — Simul. Comput. B8( 4), 369-377. 
Nagaraja, H. N. (1983). On the information contained in an order statistic. Techni- 
cal Report No. 278, Department of Statistics, The Ohio State University. 

Nagaraja, H. N. (1986a). A note on conditional Markov property of discrete order 
statistics, J. Statist. Plann. Inf. 13, 37-43. 

Nagaraja, H. N. (1986b). Structure of discrete order statistics, J. Statist. Plann. Inf. 
13, 165-177. 

Nagaraja, H. N. (1986c). Comparison of estimators and predictors from two-param- 
eter exponential distribution, Sankhya B 48, 10-18. 

Nagaraja, H. N. (1988a). Record values and related statistics — A review, Commun. 
Statist. — Theor. Meth. 17, 2223-2238. 

Nagaraja, H. N. (1988b). Selection differentials, in S. Kotz, N. L. Johnson, and C. B. 
Read, Eds., Encyclopedia of Statistical Sciences (Vol. 8, pp. 334-338), Wiley, New 
York. 

Nagaraja, H. N. (1990). Some reliability properties of order statistics, Commun. 
Statist. — Theory. Meth. 79(1), 307-316. 

Nagaraja, H. N. (1992). Order statistics from discrete distributions (with discussion), 
Statistics 23(2). 

Ogawa, J. (1951). Contributions to the theory of systematic statistics, I, Osaka Math. 
J. 3, 175-213. 

Pearson, E. S. and Chandrasekar, C. (1936). The efficiency of statistical tools and 
a criterion for the rejection of outlying observations, Biometrika 28, 308-320. 

Pearson, K. (1934). Tables of the Incomplete B-Function, Cambridge University 
Press, England. 

Pexider, J. V. (1903). Notiz iiber Funktional theoreme, Monatsh. Math. Phys. 14, 
293-301. 



268 


BIBLIOGRAPHY 


Pfeifer, D. (1982). Characterizations of exponential distributions by independent 
non-stationary record increments, /. Appl. Probab. 19, 127-135. Correction 19, 
906. 

Pinsker, I. Sh., Kipnis, V., and Grechanovsky, E. (1986). A recursive formula for 
the probability of occurrence of at least m out of N events, Am. Statist. 40, 
275-276. 

Pitman, E. J. G. (1936). Sufficient statistics and intrinsic accuracy, Proc. Cambridge 
Philos. Soc. 32, 567-579. 

Ramberg, J. S. and Tadikamalla, P. R. (1978). On the generation of subsets of 
order statistics, J. Statist. Comput. Simul. 6, 239-241. 

Rao, C. R. (1973). Linear Statistical Inference and its Applications, Wiley, New York. 

Reiss, R. D. (1989). Approximate Distributions of Order Statistics: With Applications to 
Nonparametric Statistics, Springer- Verlag, Berlin. 

Renyi, A. (1953). On the theory of order statistics, Acta Math. Acad. Sci. Hung. 4, 
191-231. 

Renyi, A. (1962). Theorie des elements saillants d’une suite d’observations, Collo- 
quium on Combinatorial Methods in Probability Theory, (pp. 104-117), Math. 
Inst., Aarhus University. 

Resnick, S. I. (1973). Record values and maxima, Ann. Probab. 1, 650-662. 

Resnick, S. I. (1987). Extreme Values, Regular Variation, and Point Processes, 
Springer-Verlag, New York. 

Romanovsky, V. (1933). On a property of the mean ranges in samples from a normal 
population and on some integrals of Professor T. Hojo, Biometrika 25, 195-197. 

Rossberg, H. J. (1972). Characterization of the exponential and the Pareto distribu- 
tions by means of some properties of the distributions which the differences and 
quotients of order statistics are subject to, Math. Operationsforschung Statist. 3, 
207-216. 

Ruben, H. (1954). On the moments of order statistics in samples from normal 
populations, Biometrika 41, 200-227. 

ROschendorf, L. (1985). Two remarks on order statistics, J. Statist. Plann. Inf. 11, 
71-74. 

Saleh, A. K. MD. E., Scott, C., and Junkins, D. B. (1975). Exact first and second 
order moments of order statistics from the truncated exponential distribution, 
Naval Res. Logist. Quart. 22, 65-77. 

Sarhan, A. E. and Greenberg, B. G. (1956). Estimation of location and scale 
parameters by order statistics from singly and doubly censored samples. Part I. 
The normal distribution up to samples of size 10, Ann. Math. Statist. 27, 
427-451. Correction 40, 325. 

Sarhan, A. E. and Greenberg, B. G. (1957). Tables for best linear estimates by 
order statistics of the parameters of single exponential distributions from singly 
and doubly censored samples, /. Am. Statist. Assoc. 52, 58-87. 

Sarhan, A. E. and Greenberg, B. G. (1959). Estimation of location and scale 
parameters for the rectangular population from censored samples, J. R. Statist. 
Soc., Ser. B. 21, 356-363. 



BIBLIOGRAPHY 


269 


Sarhan, A. E. and Greenberg, B. G. (Eds.) (1962a). Contributions to Order Statis- 
tics, Wiley, New York. 

Sarhan, A. E. and Greenberg, B. G. (1962b). The best linear estimates for the 
parameters of the normal distribution, in A. E. Sarhan and B. G. Greenberg, 
Eds., Contributions to Order Statistics (pp. 206-269), Wiley, New York. 

Scheffe, H. and Tukey, J. W. (1945). Non-parametric estimation — I. Validation of 
order statistics, Ann. Math. Statist. 16, 187-192. 

Schneider, B. E. (1978). Trigamma function. Algorithm AS121, Appl. Statist. 27, 
97-99. 

Schucany, W. R. (1972). Order statistics in simulation, J. Statist. Comput. Simul. 1, 
281-286. 

Seal, K. C. (1956). On minimum variance among certain linear functions of order 
statistics, Ann. Math. Statist. 27, 854-855. 

Serfling, R. J. (1980). Approximation Theorems of Mathematical Statistics, Wiley, 
New York. 

Shah, B. K. (1966). On the bivariate moments of order statistics from a logistic 
distribution, Ann. Math. Statist. 37, 1002-1010. 

Shah, B. K. (1970). Note on moments of a logistic order statistics, Ann. Math. 
Statist. 41, 2151-2152. 

Shapiro, S. S. and Wilk, M. B. (1965). An analysis of variance test for normality 
(complete samples), Biometrika 52, 591-611. 

Shewhart, W. A. (1931). Economic Control of Quality of Manufactured Product, 
D. Van Nostrand, Princeton, NJ. 

Shorack, G. R. and Wellner, J. A. (1986). Empirical Processes with Applications to 
Statistics, Wiley, New York. 

Shorrock, R. W. (1973). Record values and inter-record times, J. Appl. Probab. 10, 
543-555. 

Sillitto, G. P. (1951). Interrelations between certain linear systematic statistics of 
samples from any continuous population, Biometrika 38, 377-382. 

Sillitto, G. P. (1964). Some relations between expectations of order statistics in 
samples of different sizes, Biometrika 51, 259-262. 

Siotani, M. (1957). Order statistics for discrete case with a numerical application to 
the binomial distribution, Ann. Inst. Statist. Math. 8, 95-104. 

Siotani, M. and Ozawa, M. (1958). Tables for testing the homogeneity of k inde- 
pendent binomial experiments on a certain event based on the range, Ann. Inst. 
Statist. Math. 10, 47-63. 

Smirnov, N. V. (1949). Limit distributions for the term of a variational series, Trudy 
Mat. Inst. Steklov 25, 1-60 (in Russian). English translation in Am. Math. Soc. 
Transl. 67 (1952). 

Srikantan, K. S. (1962). Recurrence relations between the PDF’s of order statistics, 
and some applications, Ann. Math. Statist. 33, 169-177. 

Srivastava, R. C. (1974). Two characterizations of the geometric distribution, 
J. Am. Statist. Assoc. 69, 267-269. 



270 


BIBLIOGRAPHY 


Stephens, M. A. (1986). Tests for the exponential distribution, in R. B. D’Agostino 
and M. A. Stephens, Eds. Goodness-of-Fit Techniques (pp. 421-459), Marcel 
Dekker, New York. 

Steutel, F. W. and Thiemann, J. G. F. (1989). On the independence of integer and 
fractional parts, Statist. Neerland. 43, 53-59. 

Stigler, S. M. (1973a). Simon Newcomb, Percy Daniel), and the history of robust 
estimation 1885-1920, J. Am. Statist. Assoc. 68, 872-879. 

Stigler, S. M. (1973b). The asymptotic distribution of the trimmed mean, Ann. 
Statist. 1, 472-477. 

Stigler, S. M. (1974). Linear functions of order statistics with smooth weight 
functions, Ann. Statist. 2, 676-693. Correction 7, 466. 

Sukhatme, P. V. (1937). Tests of significance for samples of the x 2 population with 
two degrees of freedom, Ann. Eugen. 8, 52-56. 

SYSTAT (1990). Version 5.1. Systat, Evanston, IL. 

Takahasi, K. (1988). A note on hazard rates of order statistics, Commun. 
Statist. — Theor. Meth. 17, 4133-4136. 

Tarter, M. E. (1966). Exact moments and product moments of the order statistics 
from truncated logistic distribution, J. Am. Statist. Assoc. 61, 514-525. 

Tata, M. N. (1969). On outstanding values in a sequence of random variables, 
Z. Wahrsch. Verw. Gebiete. 12, 9-20. 

Teichroew, D. (1956). Tables of expected values of order statistics and products of 
order statistics for samples of size twenty and less from the normal distribution, 
Ann. Math. Statist. 27, 410-426. 

Thompson, W. R. (1936). On confidence ranges for the median and other expectation 
distributions for populations of unknown distribution form, Ann. Math. Statist. 
7, 122-128. 

Tietjen, G. L., Kahaner, D. K., and Beckman, R. J. (1977). Variances and covari- 
ances of the normal order statistics for sample sizes 2 to 50, Selected Tables in 
Mathematical Statistics 5, 1-73. 

Tiku, M. L. (1967). Estimating the mean and standard deviation from a censored 
normal sample, Biometrika 54, 155-165. 

Tiku, M. L., Tan, W. Y., and Balakrishnan, N. (1986). Robust Inference, Marcel 
Dekker, New York. 

Tippett, L. H. C. (1925). On the extreme individuals and the range of samples taken 
from a normal population, Biometrika 17, 364-387. 

von Mises, R. (1936). La distribution de la plus grande de n valeurs, Rev. Math. 
Union Interbalcanique 1, 141-160. Reproduced in Selected Papers of Richard von 
Mises, Am. Math. Soc. II (1964), 271-294. 

Wilk, M. B. and Gnanadesikan, R. (1968). Probability plotting methods for the 
analysis of data, Biometrika 55, 1-17. 

Wilks, S. S. (1962). Mathematical Statistics, Wiley, New York. 

Yang, M. C. K. (1975). On the distribution of the inter-record times in an increasing 
population, J. Appl. Probab. 7, 432-439. 

Young, D. H. (1970). The order statistics of the negative binomial distribution, 
Biometrika 57, 181-186. 



Author Index 


Abramowitz, M., 44, 82, 259 
Ahmed, S. E., 77, 260 
Ahsanullah, M., 158, 259 
Aitken, A. C„ 171, 259 
Andrews, D. F., 194, 259 
Arnold, B. C„ 119, 128, 131, 133, 252, 
259 

Balakrishnan, N., 1, 37, 38, 47, 59, 77, 
86, 87, 90, 93, 99, 100-102, 105, 
116-119, 128, 131, 134-136, 139, 
170, 173, 177, 179, 180, 252, 259, 
260, 265, 270 

Balasubramanian, K., 134, 173, 260 
Balkema, A. A., 225, 260 
Ballerini, R., 253, 260 
Barlow, R. E., 39, 260 
Barnett, V. D„ 1, 103, 190, 192, 194, 
233, 260 

Bartholomew, D. J., 170, 260 

Barton, D. E., 245, 262 

Basu, D., 90, 260 

Beckman, R. J., 87, 90, 270 

Beg, M. I., 257, 265 

Bendre, S. M„ 116, 259 

Bennett, C. A., 179, 260 

Berger, R. L., 2, 261 

Bernardo, J. M., 82, 260 

Beyer, W. H., 44, 260 

Bhattacharyya, G. K., 167, 168, 260, 266 

Bickel, P. J., 2, 194, 259, 261 


Biondini, R., 252, 261 
Birnbaum, A., 38, 261 
Bishop, Y. M. M., 3, 261 
Blom, G., 179, 261 
Boes, D. C., 2, 267 
Bondesson, L., 174, 261 
Borgan, O., 168, 261 
Bose, R. C., 88, 261 
Burr, I. W„ 59, 261 
Burrows, P. M., 37, 261 

Cadwell, J. H., 135, 261 
Casella, G., 2, 261 
Castillo, E., 1, 207, 261 
Chan, P. S., 139, 173, 260 
Chandler, K. N„ 241, 261 
Chandrasekar, C., 190, 267 
Chernoff, H., 179, 230, 261 
Chibisov, D. M„ 206, 261 
Clark, C. E., 131, 261 
Cohen, A. C„ 1, 37, 38, 90, 170, 177, 
179, 180, 260, 261 
Craig, A. T„ 2, 10, 164, 264 

D’Agostino, R. B„ 187, 189, 262 
Daniell, P. J., 206, 230, 262 
David, F. N„ 80, 128, 131, 245, 262 
David, H. A., xv, 1, 40, 90, 119, 120, 
127, 128, 131, 138, 185, 202, 222, 
262, 264 

Davis, C. S., 94, 262 
Davis, H. T„ 82, 262 


271 



272 


AUTHOR INDEX 


Davis, R. C, 163, 164, 262 
De Haan, L., 206, 209, 212, 225, 260, 
262 

Desu, M. M., 146, 262 
Devroye, L., 97, 262 
Doksum, K. A., 2, 261 
Downton, F., 132, 262 
Dudewicz, E. J., 2, 262 
Dudman, J., 38, 261 

Epstein, B., 188, 262 

Falk, M„ 227, 263 
Feller, W„ 245, 263 
Ferentinos, K. K., 195, 263 
Ferguson, T. S., 154, 263 
Fienberg, S. E., 3, 261 
Fisher, R. A., 3, 206, 209, 216, 263 
Frechet, M., 206, 209, 263 

Galambos, J., 1, 142, 153, 206, 209, 211, 
217, 219, 263 

Gastwirth, J. L., 179, 230, 261 
Gentle, J. E., 97, 265 
George, E. O., 150, 263 
Gibbons, J. D., 189, 263 
Glick, N„ 242, 263 
Gnanadesikan, R., 186, 187, 270 
Gnedenko, B., 3, 206, 211, 263 
Godwin, H. J., 90, 263 
Govindarajulu, Z., 93, 116, 133, 135, 
137, 138, 263 

Graybill, F. A., 2, 175, 263, 267 
Grechanovsky, E., 13, 268 
Greenberg, B. G., xv, 1, 92, 168, 
176-178, 198, 199, 268, 269 
Grubbs, F. E., 190, 191, 201, 263 
Grudzien, Z., 256, 263 
Gumbel, E. J., 120, 206, 264 
Gupta, A, K., 180, 264 
Gupta, S. S., 38, 68, 69, 82, 86, 88, 100, 
139, 189, 261, 264 

Hall, P„ 216, 264 
Halperin, M., 168, 264 
Hamming, R. W., 246, 264 


Hampel, F. R., 194, 259 
Harter, H. L„ 1, 87, 128, 264 
Hartley, H. O., 95, 96, 104, 120, 264, 
266 

Hawkins, D. M„ 192, 264 
Hochberg, Y., 189, 264 
Hoeffding, W., 143, 264 
Hogg, R. V., 2, 10, 164, 193, 230, 264 
Holland, P. W., 3, 261 
Horn, P. S., 96, 104, 265 
Houchens, R. L., 249, 265 
Huang, J. S., 36, 145, 265 
Huber, P. J., 194, 259, 265 
Huzurbazar, V. S., 163, 265 

Johns, M. V., Jr., 179, 230, 261 
Johnson, N. L., 80, 128, 131, 262 
Johnson, R. A., 167, 266 
Joshi, P. C., 73, 93, 97-99, 102, 103, 
113, 119, 134, 136, 138, 260, 262, 
265 

Junkins, D. B., 98, 268 

Kabir, A. B. M. L„ 158, 259 
Kagan, A. M., 142, 265 
Kahaner, D. K., 87, 90, 270 
Kale, B. K., 202, 265 
Kaminsky, K. S., 182, 265 
Kennedy, W. J., 97, 265 
Khatri, C. G., 203, 265 
Kipnis, V., 13, 268 
Kirmani, S. N. U. A., 257, 265 
Kochar, S. C., 256, 265 
Kocherlakota, S., 101, 260 
Korwar, R. M., 254, 265 
Kotz, S., 142, 153, 263 
Krishnaiah, P. R., 133, 266 
Kuczma, M., 149, 266 

Lai, T. L„ 252, 266 

Laurent, A. G., 1%, 266 

Lawless, J. F„ 162, 168, 183, 200, 266 

Leadbetter, M. R., 206, 236, 266 

Lehmann, E. L, 163, 266 

Lewis, T., 1, 190, 192, 194, 233, 260 

Lieblein, J., 37, 266 



AUTHOR INDEX 


273 


Likes, J., 182, 266 
Lindgren, G., 206, 236, 266 
Lingappaiah, G. S., 200, 266 
Linnik, Yu. V., 142, 265 
Lloyd, E. H„ 159, 171, 266 
Ludwig, O., 121, 122, 266 
Lurie, D., 95, 96, 104, 266 

Malik, H. J„ 77, 86, 87, 116, 118, 134, 
135, 259, 260 
Malmquist, S., 80, 266 
Mason, D. M., 228, 266 
Mason, R. L., 95, 266 
Mehrotra, K. G., 167, 170, 266 
Melnick, E. L„ 71, 266, 267 
Mishra, S. N„ 2, 262 
Mood, A. M., 2, 267 
Moriguti, S., 120, 267 
Murphy, R. B„ 102, 192, 234, 267 

Nagaraja, H. N„ 39-41, 50, 58, 106, 
167, 182, 234, 242, 251, 255, 267 
Nanda, P., 170, 266 

Ogawa, J., 180, 267 
Olkin, 1., 189, 263 
Ozawa, M., 70, 269 

Panchapakesan, S., 68, 69, 189, 264 
Pearson, E. S., 190, 267 
Pearson, K., 13, 267 
Pexider, J. V., 158, 267 
Pfeifer, D„ 253, 267 
Pinsker, I. Sh., 13, 267 
Pitman, E. J. G., 163, 268 
Proschan, F., 39, 260 

Ramberg, J. S., 96, 268 
Rao, C. R„ 142, 152, 179, 265, 268 
Reiss, R. D., 207, 268 
Renyi, A., 73, 248, 268 
Resnick, S. I., 206, 219, 241, 244, 253, 
260, 268 

Rhodin, L. S„ 182, 265 
Rizvi, M. H„ 133, 266 
Robbins, H., 252, 266 
Rogers, W. H„ 194, 259 


Romanovsky, V., 135, 269 
Rootzen, H„ 206, 236, 266 
Rossberg, H. J., 143, 152, 153, 158, 268 
Rousseau, C. C., 150, 263 
Ruben, H„ 87, 268 
Riischendorf, L., 50, 268 

Saleh, A. K. Md. E„ 98, 268 
Sarhan, A. E., xv, 1, 92, 168, 176-178, 
198, 199, 268, 269 
Scheffe, H„ 22, 269 
Schlipf, J. S„ 96, 104, 265 
Schneider, B. E., 82, 269 
Schucany, W. R., 95, 269 
Scott, C„ 98, 268 
Seal, K. C„ 137, 269 
Serfling, R. J., 207, 228, 269 
Shah, B. K„ 38, 82, 86, 100, 264, 269 
Shapiro, S. S., 189, 269 
Shewhart, W. A., 185, 269 
Shorack, G. R., 207, 228, 269 
Shorrock, R. W., 241, 269 
Shu, V. S„ 40, 202, 262 
Siddiqui, M. M., 252, 261 
Sillitto, G. P., 132, 135, 269 
Sinha, S. K„ 202, 265 
Siotani, M., 70, 269 
Smirnov, N. V., 206, 225 , 269 
Sobel, M„ 189, 263 
Srikantan, K. S„ 112, 116, 132, 269 
Srivastava, R. C., 60, 269 
Stegun, 1. A., 44, 82, 259 
Stephens, M. A., 94, 188, 262, 269 
Steutel, F. W., 77, 270 
Stigler, S. M„ 206, 228, 231, 232, 270 
Sukhatme, P. V., 72, 203, 270 
Szynal, D., 256, 262 

Tadikamalla, P. R., 96, 268 
Takahasi, K., 39, 270 
Tamhane, A. C., 189, 264 
Tan, W. Y„ 170, 270 
Tarter, M. E., 101, 270 
Tata, M. N„ 244, 270 
Teichroew, D., 91, 270 
Thiemann, J. G. F., 77, 270 
Thompson, W. R., 185, 270 



274 


AUTHOR INDEX 


Tietjen, G. L., 87, 90, 270 
Tiku, M. L., 170, 270 
Tippett, L. H. C., 3, 87, 206, 209, 216, 
263, 270 

Tukey, J. W„ 22, 187, 194, 259, 269 
von Mises, R., 206, 212, 270 


Wellner, J. A., 207, 228, 269 
Wilk, M. B., 186, 187, 189, 269, 270 
Wilks, S. S., 60, 61, 270 
Williams, G. T., 131, 261 

Yang, M. C. K., 253, 270 
Young, D. H., 133, 270 



Subject Index 


adaptive estimates, 193-194 
admissible matrix, 133 
allocation of prizes, 7 
almost sure convergence, 205 
annual rainfall, 257 
approximations for cumulants, 80 
ascending method, 95, 96 
asymptotic distributions, 205-238, 244 
of central and intermediate order 
statistics, 223-227 
of exponential extremes, 207-208 
of extremes from discrete 
distributions, 216-217 
joint distributions, 219-222 
of L statistics, 179, 227-234 
of maxima and minima 208-216 
asymptotic independence of extremes, 
221 

Basu’s theorem, 90, 91, 93 

Bernoulli distribution, 63-65 

best linear unbiased estimate (BLUE), 

159, 171-175 

asymptotic approaches, 179-180 
for exponential distribution, 169, 
176-177 

for normal distribution, 177-179 
for uniform distribution, 168, 175, 176 
best linear unbiased predictor (BLUP), 

160, 181, 182, 199 
binomial distribution, 59, 66-70 


Cantelli’s inequality, 247 
Cauchy distribution, 103, 163, 193, 237 
Cauchy equation, 152, 158 
Cauchy-Schwarz inequality, 120, 121, 
122, 123, 124, 125, 250 
censored sampling, 4, 159, 178, 179 
censoring 
progressive, 162 
random, 161 
Type I, 160, 167, 169 
Type II, 161, 167, 168, 174, 181, 195, 
197, 199, 203 

central limit theorem 205, 247 
central order statistics, 206, 223-226 
characterizations, 7, 141-158, 188 
based on moments of order statistics, 
143-152 

geometric distribution, 60 
involving dependency assumptions, 
153-155 

involving samples of random size, 
155-157 

complete families of polynomials, 144 
complete sufficient statistic, 90, 91, 93, 
164, 188 

computer simulation, 95-97 
computing checks, 118, 135 
conditional distribution of order 
statistics, 23-26 

confidence intervals, 162, 164, 183-185, 

200, 202 


275 



276 


SUBJECT INDEX 


convergence in distribution, 205 
convergency in probability, 205 
Cramer-Rao inequality, 165 
cross-cumulants, 131 
culling, 6 

cumulant generating function, 82 
cumulants, 131 

decreasing failure rate (DFR), 38, 39, 
59 

decreasing failure rate average (DFRA), 
39 

deFinetti theorem, 251 
delta method, 3 
dependent sequences, 251 
descending method, 95, 96 
discard-deviation, 230 
discordant observations, 189 
discrete order statistics, 41-61, 216-217 
dependence structure, 48-50 
geometric distribution, 51-53 
joint probability mass function, 45-48 
range, 50-51 

sampling without replacement, 53-56 
discrete Pareto distribution, 217 
discrete uniform distribution, 43, 51, 55, 
58, 59 

distribution free intervals, 183-186 
domain of attraction, 208, 211, 213 
double-exponential distribution, 104 

empirical c.d.f., 187, 189 
empirical quantile function, 7 
Euler’s constant, 246 
exceedance, 236 

exchangeable random variables, 54, 138, 
251,252 

exponential distribution 
asymptotic distribution of extremes, 
207-208, 215, 235 

best linear unbiased estimates, 176, 
177, 196—198 

censored sample, 168, 169 
characterizations, 7, 141, 146-148, 
150-155, 158 

distribution of order statistics, 11, 34, 
35, 36, 40, 72-78, 103 
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likelihood ratio test, 188, 203 
outliers, 202 

prediction of order statistics, 181, 182 
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simulation, 95, 96 
spacings, 11, 106 
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extreme order statistics, 3 
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moments, 44 
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robust estimation, 192 
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extreme processes, 241 
extreme value distribution, 138, 210 
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folded distribution, 137 
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gamma function, 15 
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geometric distribution 
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graphical procedures, 186-187 

half logistic distribution, 104-106 
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inequality measurement, 6 
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interrecord time, 242, 248, 253 
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likelihood ratio test, 188, 190, 203 
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linear regression, 155 
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log-logistic distribution, 156 
log-normal distribution, 187 
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masking effect, 191 
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maximum likelihood estimate (MLE), 
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generating function, 81 
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monotone hazard rate, 157 
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multiple outliers, 191 
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normal distribution 
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Q-Q plots, 187 
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order statistics, 1, 9 
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Pareto distribution, 35, 36, 195, 215, 
237, 252, 255 

penultimate limit behavior, 216 
permutation group, 159 
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pivotal quantity, 195, 200 
Poisson distribution, 70-71, 235 
power function distribution, 14, 20, 28 
prediction, 180-183 
prediction interval, 182, 200 
probability integral transformation, 189 
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quality control, 5 
quantile differences, 185 
quantile function, 2, 109 
quantiles, 183-185 
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Rayleigh distribution, 197, 235 
record counting process, 242, 246-247 
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record range, 248-249 
record time, 242, 245-248 
record value, 220, 241-257 
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reliability, 5 

robust estimates, 4, 192-194 

samples of random size, 155-157 
sampling without replacement, 53-56 
scale estimates, 171-180 
Schroder’s equation, 149, 156 
selection, 6, 189 
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series approximation, 128-131 
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Shapiro-Wilk test, 189 

simulation, 95-97 

sine distribution, 37 
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Slutsky’s theorem, 224 

smallest order statistic, 12, 95 
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survival curves, 141 
symmetric distributions, 26, 119, 123, 
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174, 187 

tail thickness, 230 
three point distribution, 65-66 
tolerance limits, 185-186, 200 
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triangle rule, 112, 115, 144 
triangular distribution, 37 
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truncated exponential distribution, 
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truncated logistic, 100, 101 
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ultimate limit behavior, 216 
unbiased estimate, 182 
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uniform distribution 
asymptotic distribution of order 
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censored sample, 195, 199 
characterizations, 150 
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expected maximum, 126, 127 
median, 27 
midrange, 33, 34 
moments of order statistics, 131 
probability integral transformation, 
21, 189 
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record range, 249 
record values, 249, 255 
simulation, 95, 104, 106 
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variational sequence, 1 
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weak convergence, 205, 206 
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